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ABSTRACT 

RESULTS  OF  A  SURVEY  ON  MANAGEMENT  TECHNIQUES  AND  PROCEDURES 
USED  IN  SOFTWARE  DEVELOPMENT  PROJECTS  BY  THE  US  AEROSPACE  INDUSTRY 

BY 

\  Richard  H.  Thayer  and  John  H.  Lehman 

This  report  contains  the  results  of  a  survey  conducted  in  1977  and  1978 
on  how  the  US  Aerospace  Industry  manages  its  software  development  projects. 
The  sample  of  the  US  Aerospace  Industry  surveyed  was  those  companies  who 
had  a  membership  in  the  AIAA  Technical  Committee  on  Computer  Systems. 

These  committee  members  represented  47  major  corporations  or  major 
corporate  subdivisions  and  occupied  top  positions  in  software  management 
within  their  firms. 

The  survey  used  a  questionnaire  containing  225  numbered  questions  on 
project  management.  By  using  "multiple  choice-multiple  answer"  questions, 
approximately  1,328  separate  responses  were  possible.  The  survey  was 
divided  into  three  parts.  Part  One  dealt  with  defining  the  organization, 
management  structure,  and  philosophy  of  the  firm.  This  part  was  intended 
to  be  answered  by  top  management  to  provide  the  backdrop  against  which 
the  individual  projects  would  be  viewed.  Part  Two  concerned  individual 
software  development  projects  and  was  intended  to  be  completed  by  the 
project  manager.  Part  Three  was  designed  to  obtain  the  opinions  and 
perceptions  of  software  development  project  managers  on  major  issues  and 
major  problems  of  software  engineering  project  management. 

This  paper  reports  the  results  from  Part  Two;  that  portion  of  the 
questionnaire  dealing  with  actual  projects.  Parts  One  and  Three,  the 
portions  dealing  with  the  company  environment  and  software  development 
problems,  are  reported  on  in  Volumes  I  and  III. 

The  answers  from  the  surveyees  were  abbreviated  or  coded  and  tabulated 
in  this  report.  In  addition,  a  portion  of  the  narrative  answers  to  the 
survey  were  also  reported.  To  protect  the  participants,  all  references 
to  individuals  or  their  companies  have  been  eradicated.  This  report 
does  not  attempt  to  analyze  or  come  to  conclusions  about  the  data  only  to 
report  it  as  clearly  as  possible. 
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SECTION  1 
RESULTS 

BACKGROUND 

The  survey  in  the  title  of  this  paper  was  a  survey  of  the  US  Aero¬ 
space  Industry  to  determine  what  management  techniques  and  procedures 
are  used  by  the  aerospace  industry  in  their  software  development  projects. 

This  survey  was  designed,  written,  tested,  and  implemented  by  the 
authors  in  the  spring  and  summer  of  1977.  This  survey  was  accomplished 
to  collect  data  for  the  preparation  of  a  paper  on  software  engineering 
project  management  which  was  presented  at  the  American  Institute  of 
Aeronautics  and  Astronautics  (AIAA)  Conference,  Computers  in  Aerospace. 

31  Oct  -  2  Nov  1977. 

That  portion  (or  sample)  of  the  US  Aerospace  Industry  that  was 
surveyed  were  those  companies  who  had  a  membership  in  the  AIAA  Techni¬ 
cal  Committee  on  Computer  Systems  (who  were  the  host  to  the  conference 
on  Computers  in  Aerospace) .  These  committee  members  represented  47  major 
aerospace  corporations  or  major  corporate  subdivisions  and  occupied  top 
positions  in  software  management  within  their  firms.  These  committee 
members  were  in  an  ideal  position  to  report  on  how  the  US  Aerospace 
Industry  manages  its  software  development  projects. 

Initial  contact  was  made  in  May  1977  to  determine  which  members  of  the 
committee  would  be  interested,  willing,  and  able  to  participate.  Forty- 
five  members,  representing  35  companies,  agreed  to  respond.  The  initial 
draft  of  the  survey  was  completed  in  June  1977  and  was  critiqued  by 
approximately  25%  of  the  total  committee  membership.  The  results  of  this 
critique,  along  with  ocher  corrections,  were  incorporated  into  the  final 
survey.  The  survey  was  mailed  10  August  1977.  On  6  September  1977,  with 
29  of  the  completed  surveys  on  hand,  the  authors  wrote  the  first  report 
for  the  proceedings  of  the  Computers  in  Aerospace  conference.  This  paper 
can  be  found  in  the  conference  proceedings,  A  Collection  of  Technical 
Papers.  By  the  time  the  presentation  was  given  on  1  November  1977,  55 
projects  were  reported  on,  representing  33  companies  or  70%  returns.  The 
companies  surveyed  were  predominantly  aerospace  firms,  with  government 
contracts,  reporting  on  large  to  very  large  projects.  The  presentation 


given  by  author  Richard  H.  Thayer  at  the  conference  (called  Report  Nr  2, 
AIAA  Project  Management  Survey)  used  the  more  complete  data  and  a 
different  approach. 

However,  the  survey  did  not  end  there.  More  completed  forms  still 
trickled  in  until  by  the  spring  of  1978,  62  projects  had  been  reported 
on,  representing  37  firms  for  a  76%  return  rate.  This  later  Increased 
to  66  by  the  end  of  the  summer  1978  for  86%  return  rate  (see  Appendix  A 
for  a  list  of  participants).  Because  of  this  extensive  participation, 
a  decision  was  made  by  the  AlAA  Technical  Committee  on  Computer  Systems 
to  make  further  use  of  the  data  by  writing  an  assessment  paper  on  the 
state-of-the-art  in  software  development  project  management.  Mr.  Gene 
F.  Walters,  General  Electric  Company,  Command  and  Information  Systems, 
and  Mr.  Jack  E.  Bloodworth,  Boeing  Aerospace  Company,  were  given 
primary  responsibility  for  this  paper. 

In  order  to  meet  these  goals,  the  Rome  Air  Development  Center,  the 
Sacramento  Air  Logistics  Center,  and  The  Boeing  Aerospace  Company 
offered  their  services,  and  in  some  cases  the  services  of  their  company's 
data  processing  capability,  to  reduce  and  analyze  the  data. 

The  remaining  problem  was  to  reduce  the  data  into  a  form  useable  by  a 
computer.  This  involved  "coding"  the  narrative  portions  and  free  form 
answers  of  the  survey,  verifying  that  the  answers  were  consistent,  and 
abbreviating  all  "fill  in  the  blank"  answers.  This  was  accomplished  by 
one  of  the  architects  of  the  survey,  Richard  H.  Thayer. 

PURPOSE  OF  SURVEY 

The  purpose  of  this  survey  was  to  view  a  sample  of  the  US  Aerospace 
Industry  through  the  use  of  a  questionnaire  to  determine  how  this 
industry  managed  their  software  development  projects.  Specifically,  the 
questions  the  survey  attempted  to  answer  were  as  follows; 

1.  What  are  the  current  practices  in  Software  Engineering 
Project  Management  today? 

2.  Are  the  new  developments  in  management,  i.e.,  "modem" 
management  techniques  or  project  management  techniques, 
being  used? 

What  are  the  trends  in  the  Software  Engineering  Project 
Managemenr.? 


3. 


What  are  the  relationships  between  Software  Engineering 
Project  Management  techniques  and  successful  delivery  of 
software? 

What  are  the  relationships  between  various  elements  of 
Software  Engineering  Project  Management  as  a  system? 

What  are  the  relationships  between  "modern"  Software 
Engineering  techniques  and  Software  Engineering  Project 
Management? 


THL  SURVEY 


The  approach  taken  l-n  answering  the  questions  was  first  to  design  a 
model  of  software  engineering  project  management  as  a  system,  define  the 
elements  of  that  model,  and  hypothesize  relationships  between  those 
elements.  Second,  was  to  form  a  questionnaire  around  this  model  using  the 
various  elements  or  variables  of  the  model  as  questions  and  possible 
answers  (see  Appendix  B  for  a  copy  of  the  questionnaire) .  The  survey 
used  a  rather  lengthy  questionnaire  containing  225  numbered  questions. 
Beyond  that,  by  using  "multiple  choice-multiple  answer"  questions 
approximately  1,328  separate  responses  were  possible. 

The  72  page  survey  was  divided  into  3  parts.  Part  One  dealt  with 
defining  the  total  organization,  management  structure,  and  philosophy 
of  the  firm.  This  part  was  intended  to  be  answered  by  top  management 
to  provide  the  backdrop  against  which  the  individual  projects  would  be 
viewed.  Part  Two  concerned  individual  projects  and  was  intended  to  be 
completed  by  the  project  manager.  Part  Three  consisted  of  general 
questions,  not  project  specific,  calling  for  evaluation,  opinions,  and 
suggestions  on  the  major  problems  of  software  engineering  project  manage¬ 
ment.  Part  Three  was  also  intended  to  be  completed  by  a  project  manager. 
PURPOSE  OF  THIS  REPORT 

This  report  was  prepared  as  a  means  of  (1)  reducing  the  answers  to 
Part  Two  and  Questions  1,  2,  3,  4,  and  25  of  Part  Three  of  the  question¬ 
naire  in  "raw  form"  so  that  they  could  be  computerized,  and  (2) 
providing  the  answers  to  the  questionnaire  to  satisfy  the  many  requests 
from  the  computer  community  for  access  to  the  data  collected  as  a 
result  of  this  survey.  (The  answers  to  Part  One  and  the  remainder  of 
Part  Three  are  provided  in  Volumes  I  and  III.)  Because  of  the 

restrictions  placed  by  the  participants  on  the  use  of  their  submissions, 
the  survey  forms  with  the  complete  answers  cannot  be  distributed.  This 
report  was  selected  as  a  means  of  capturing  and  documenting  as  much  of 
the  raw  data  as  possible  without  revealing  the  source  of  the  inform¬ 
ation.  In  essence,  this  report  does  not  contain  "raw  data,"  but 
reduced  data  in  abbreviated  and  coded  form  which  will  effectively  dis¬ 
guise  the  participant  but  still  allow  interested  computer  scientists  to 
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use  the  data  for  their  own  requirements. 

This  report  does  not  attempt  to  analyze  or  come  to  conclusions  about 
the  data,  only  to  report  it  as  clearly  as  possible.  Only  minimum 
interpretation  was  made  to  enable  the  answers  to  be  tabulated  for 
eventual  analysis .  Although  66  projects  were  reported,  the  authors 
removed  six  projects  that  did  not  seem  to  fit  the  norm,  leaving  a 
set  of  60  projects. 

CONTENTS  OF  THIS  REPORT 

As  already  stated,  the  purpose  of  this  report  is  not  to  analyze  the 
data  from  the  AIAA  Project  Management  Survey,  but  to  report  it  as  simply 
and  accurately  as  possible,  and,  to  keep  within  the  original  ground  rules 
of  maintaining  anonymity  of  the  participants.  Section  2  contains  the 
questions  and  answers  to  this  survey  and  Section  3  contains  cited  refer¬ 
ences.  The  participants  in  the  survey  are  listed  in  Appendix  A. 

Because  of  its  length  a  duplicate  copy  of  the  questionnaire  is  in 
Appendix  B.  The  purpose  of  this  duplicate  set  is  to  allow  the  reader  to 
quickly  peruse  the  questions  and  possible  answers  in  order  that  he  can 
determine  what  type  of  material  is  covered. 

Appendix  C  contains  the  abbreviations  used  in  reporting  the  narrative 
portions  of  this  survey  as  well  as  comments  on  the  answers  themselves. 
These  comments  pertain  primarily  to  such  things  as  accuracy  of  the 
answer,  relationship  between  questions,  and  procedures  used  in  con¬ 
triving  missing  answers.  Since  the  reduction  of  comments  to  code 
destroyed  some  of  the  richness  of  prose,  the  authors  felt  it  worthwhile 
to  Include  the  actual  written  answer  to  certain  specific  questions. 

These  answers  are  reproduced  in  Appendix  D.  To  maintain  the  concept  of 
protecting  the  participants  identity,  the  narrative  answers  cannot  be 
tied  to  any  particular  participant  in  Section  2. 

THE  FUTURE 

This  survey  is,  as  far  as  the  authors  can  determine,  the  first  at¬ 
tempt  to  query  industry  on  such  a  large  scale  on  how  software  engineering 
projects  are  managed.  A  look  at  the  list  of  contributors  in  Appendix  A 
will  attest  to  the  importance  of  this  base  of  answers.  The  tremendous 
volume  of  questions  answered  and  the  excellence  of  the  responses  dictate 
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that  this  data  be  utilized  either  in  whole  or  part  as  a  reference  for 
other  papers,  reports,  possible  texts,  and  other  technical  public¬ 
ations  for  the  benefit  of  the  US  Aerospace  Industry  and  the  data 
processing  connnunity.  The  AIAA  Technical  Conmittee  on  Computer  Systems 
is  anticipating  the  preparation  of  an  assessment  paper  on  how  industry 
manages  its  software  engineering  projects.  This  committee  welcomes 
suggestions  from  the  computing  and  aerospace  communities  on  how  to  best 
use  this  data  for  the  benefit  of  all.  All  suggestions  should  be  sent 
to  either: 

Mr.  Gene  F.  Walters  Mr.  Jack  E.  Bloodworth 

Mgr,  Software  Technologies  Mgr,  ALCM  Software 

Information  Systems  Programs  The  Boeing  Aerospace  Company 

General  Electric  Company  MS-45-70 

450  Persian  Drive  P.O.  Box  3999 

Sunnyvale,  CA  94086  Seattle,  WA  98124 

(408)  734-4980  (206)  655-6718 

The  Rome  Air  Development  Center  (RADC)  has  contracted  with  ITT 
Research  Institute  (IITRI)  to  establish  and  operate  a  software  inform¬ 
ation  analysis  center.  The  center  has  been  named  the  Data  and  Analysis 
Center  for  Software  (DACS) .  One  of  the  functions  of  DACS  is  to  acquire 
and  analyze  data  gathered  during  the  various  phases  of  the  software 
development  process  with  the  purpose  of  identifying  and  quantifying 
those  factors  which  contribute  to  the  production  of  quality  software. 

The  data  from  this  survey  has  been  contributed  to  DACS  and  is  available 
for  analysis  by  any  member  of  the  AIAA  Technical  Committee  on  Computer 
Systems  as  well  as  the  general  computer  community.  Personnel  interested 
in  receiving  copies  of  this  data,  or  requesting  analysis  of  this  data 
should  contact: 

Ms .  Lorraine  Duvalle 

Data  &  Analysis  Center  for  Software 

RADC/ISl 

Griffiss  AFB,  NY  13441 
(315)  336-0937 
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ATTACHMENT  1  TO  SECTION  1 
RELATIONSHIPS  BETWEEN  REPORTS 

The  survey  was  comprised  of  three  parts:  Part  One,  Part  Two,  and 
Part  Three.  Each  dealt  with  a  separate  facet  of  software  engineering 
project  management.  Part  One  dealt  with  the  firm  and  the  environment 
in  which  the  project  was  done.  Part  Two  devoted  itself  to  specific  soft¬ 
ware  engineering  projects  accomplished  within  the  firm.  Part  Three  asked 
the  project  managers  their  opinions  about  project  management.  Each  of 
these  parts  can,  and  does,  stand  alone.  Part  One  (results  reported  in 
Volume  I)  reports  on  the  financial  status,  organization,  management 
policies,  staffing  techniques  and  project  controls  of  the  companies  that 
report  on  the  projects  in  Part  Two. 

Part  Two  (results  reported  in  Volume  II)  reports  on  a  series  of  in¬ 
dependent  projects  which  could  be  considered  case  studies  on  how  the 
aerospace  industry  is  managing  projects  today.  Part  Three  (results 
reported  in  Volume  III)  concerns  ideas  and  perceptions  about  software 
engineering  project  management  and  does  not  relate  to  a  given  project  or 
company . 

At  the  same  time,  there  is  a  relationship  between  these  reports.  For 
instance,  you  might  be  interested  in  knowing  about  the  company  environ¬ 
ment  that  produced  the  project.  In  that  case,  you  would  want  to  know 
which  company  reported  on  which  projects.  You  might  be  interested  In 
knowing  the  background  and  project  environment  which  created  the  ideas 
and  assumptions  made  in  answering  Part  Three.  In  that  case,  you  would 
want  to  know  the  relationship  between  the  man  who  reported  the  percept¬ 
ions,  ideas  and  problems  associated  with  software  engineering  project 
management  and  the  project  he  worked  on.  To  this  aim  Table  1  tells  the 
relationships  between  Volumes  I,  II  and  III  of  this  report. 

Each  survey  answer  was  given  a  number  for  tracking  purposes.  Within 
this  number,  of  course,  is  a  Part  One,  Part  Two,  ana  Part  Three.  Since 
one  Part  One  might  cover  a  number  of  Part  Two's,  and  the  Part  Three  may 
or  may  not  be  answered  by  the  same  person  that  answered  Part  Two,  the 
relationship  between  these  is  contained  in  the  table.  Each  company  was 
given  a  two-digit  identifier,  each  project  was  given  a  three-digit 
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identifier,  and  each  answer  to  Part  Three  was  given  a  three-digit  identi¬ 
fier.  To  determine  the  common  company  relationship  between  projects, 
look  up  the  common  company  identifier  in  Table  1.  To  determine  whether 
or  not  the  same  person  answered  Part  Two  and  Part  Three,  look  under  the 
column  that  identifies  Part  Three.  If  the  entry  is  "yes”,  the  same  man 
wrote  Part  Two  and  Part  Three.  If  the  entry  is  "no",  a  different 
individual  answered  each  part. 

Projects  301  through  306  have  been  deleted  for  this  report  due  to 
size  or  incompleteness. 
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TABLE  1 

(ATTACHMENT  1  TO  SECTION  1) 
RELATIONSHIPS  OF  PROJECTS  .vTPORTED  IN  AIAA 
PROJECT  MANAGEMENT  SURVEY 
VOLUMES  I,  II  and  III 

Survey  VOL  I  VOL  II 

Identification  Nr  (1)  (Part  One)  (2) _ (Part  Two)  (3) 


101 

30 

101 

102 

30 

102 

103 

30 

103 

104 

31 

104 

105 

33  (8) 

105 

106 

34  (8) 

106 

107 

35 

107 
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35 
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109 

35 

109 
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110 

111 
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111 

112 
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113 
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113 

114 
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114 
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69 
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116 

42 

116 

117 

43 
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45 
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119 

45 

119 
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51 
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121 

66  (5) 

121 

122 

51 

122 

123 

51 

123 

124 

51 

124 

125 

52 

125 

126 

55 

126 

127 

None 

127 

128 

59 

128 

129 

None 

129 

130 

31 

130 

201 

67 

201 

VOL  III 

(Part  Three)  (4) 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
No 

None 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

Yes 


None 
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(1) 

Survey 

Identification  Nr 

(2) 

VOL  I 
(Part  One) 

(3) 

VOL  II 
(Part  Two) 

(4) 

VOL  III 
(Part  Three) 

202 

27  (7) 

202 

Yes 

203 

28  (7) 

203 

Yes 

204 

29 

204 

Yes 

205 

32 

205 

Yes 

206 

37 

206 

Yes 

207 

37 

207 

Yes 

208 

38 

208 

Yes 

209 

43 

209 

Yes 

210 

44 

210 

Yes 

211 

46  (10) 

211 

Yes 

212 

47  (10) 

212 

Yes 

213 

49 

213 

Yes 

214 

49 

214 

Yes 

215 

49 

215 

Yes 

216 

49 

216 

Yes 

217 

50 

217 

Yes 

218 

53  (11) 

218 

Yes 

219 

54  (11) 

219 

Yes 

220 

56 

220 

Yes 

221 

57 

221 

Yes 

222 

60 

222 

Yes 

223 

60 

223 

Yes 

224 

58 

224 

Yes 

225 

58 

225 

Yes 

226 

58 

226 

Yes 

227 

61 

227 

Yes 

228 

61 

228 

Yes 

229 

64 

229 

Yes 

230 

68 

230 

Yes 

301 

26  (6) 

301 

Yes 

302 

48  (10) 

None 

Mone 

303 

25  (5) 

None 

None 

304 

68 

304  (12) 

None 

305 

62 

None 

None 

306 

63  (7) 

None 

Yes 
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FOOTNOTES  FOR  TABLE  1 

(1)  Column  1  -  This  column  lists  the  returned  surveys  accordig?  to 
a  randomly  assigned  identification  number. 

(2)  Column  2  -  The  company  identification  number  is  listed  in 
column  2  and  is  used  in  Vol  I  to  describe  the  company  and  environment 
for  the  project  reported  in  Vol  II.  Projects  with  the  same  company 
number  are  from  the  same  company  and/or  major  subdivision  and  were 
completed  by  a  single  member  of  the  company.  In  other  cases,  the  same 
company  was  reported  on  by  two  or  more  individuals.  This  was  caused  by 
two  or  more  project  managers,  who  reported  on  different  projects  within 
the  same  company.  Most  of  the  time  these  "double"  reports  were  the  same. 
Comments  along  these  lines  are  contained  in  foot  notes  (5)  through  (12) . 

(3)  Column  3  -  This  column  lists  the  project  numbers  reported  in 
Vol  II. 

(4)  Column  4  -  Vol  III  reports  on  data  from  Part  Three.  This 
column  indicates  whether  or  not  the  same  person  reported/wrote  Part  Two 
and  Part  Three  of  the  survey.  This  is  done  so  that  the  reader  can  know 
if  there  is  any  relationship  between  the  project  reported  on  in  Part  Two 
and  the  surveyee's  opinions  on  the  major  problems  of  software  develop¬ 
ment  management.  Source  of  this  information  is  personal  knowledge  of 
the  author,  comparison  of  handwriting,  color  of  marker,  and  correspond¬ 
ence  with  the  surveyee. 

(5)  Company  25  and  66  are  the  same. 

(6)  Very  small  company. 

(7)  Company  27,  28  and  63  are  the  same.  Answers  reported  under 
company  28  looked  to  be  the  most  accurate  and  complete. 

(8)  Company  33  and  34  are  the  same.  Answers  reported  under  company 
33  looked  to  be  the  most  accurate  and  complete. 

(9)  Company  39  and  40  are  the  same  and  have  identical  answers. 

(10)  Company  46,  47  and  48  are  the  same.  Answers  reported  under 
company  46  are  considered  to  be  the  official  answers  by  the  surveyee. 

(11)  Company  53  and  54  are  the  same.  Answers  reported  under  company 
54  looked  to  be  the  mose  accurate  and  complete. 

(12)  Project  reported  under  project  304  was  too  large  to  be  included. 
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SECTION  2 
THE  DATA 

INTRODUCTION 

This  section  reports  on  the  actual  data  submitted  by  the  participants 
on  sixty  aerospace  projects.  It  is  reported  in  tabulated,  abbreviated 
and  coded  form  and  cannot  be  used  completely  without  Appendix  C.  Every 
effort  was  made  to  disguise  the  contributor,  including  the  deletion  of 
some  revealing  data. 

The  questionnaire  contained  many  different  styles  of  questions:  true 
or  false,  multi-choice  answers,  multi-part  questions,  f ill-in-the-blarks , 
and  narrative.  Despite  this  multitude  of  styles,  a  common  method  of 
reducing  and  reporting  the  answers  was  developed  (see  Appendixes  B  and  C) . 
Multi-part  questions  were  broken  into  separate  questions  through  the  use 
of  part  numbers  (i.e.,  01,  02,  03,  etc.)  and  sub-part  designator  (a,  b, 
c,  d,  etc.,  and  001,  002,  003,  etc.). 

Each  question  is  handled  separately  and  reported  as  an  array.  The 
horizontal  indices  of  the  array  refer  to  anonymous  project  identific¬ 
ation  numbers  (see  Section  1  for  further  explanation) .  The  vertical 
indices  refer  to  the  question,  part,  and  subpart  number.  Every  narra¬ 
tive  answer  has  been  coded  or  abbreviated  by  a  three-character  alpha¬ 
numeric  (see  Appendix  C  for  further  explanation  of  codes) . 

Generally  speaking,  the  printing  of  a  three-character  alphanumeric 
opposite  the  subpart  of  a  question  indicates  that  the  participant 
answered  "yes"  or  "true"  as  it  applies  to  that  part  of  the  question. 

If  a  given  question  has  a  "blank"  for  an  answer  this  indicates  the 
surveyee  answered  "no"  or  "false"  as  pertains  to  that  part  of  the 
question.  With  the  exception  of  "none"  or  "missing"  the  alphanumeric  is  a 
code  or  abbreviation  of  a  text  answer  that  m  iifies  the  "yes"  answer. 

The  interpretation  or  meaning  of  the  codes  can  be  found  in  Appendix  C. 

The  authors  made  every  attempt  to  use  codes  that  were  easy  to  recog¬ 
nize  (mnemonic). 

The  questionnaire  as  printed  in  this  report  is  a  modified  version  of 
the  questionnaire  as  originally  answered  (see  INTRODUCTION  TO  APPENDIX  B, 
Questionnaire  for  explanation). 
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SECTION  3 
REFERENCE 

INTRODUCTION 

In  preparing  this  survey,  literally  hundreds  of  books,  articles,  and 
papers  were  read.  The  results  of  this  literature  search  became  a 
general  model  of  how  software  .ingineering  project  management  was  accomp¬ 
lished.  This  model  is  represented  by  the  original  question..^ ire  (see 
Appendix  B).  It  would  be  impractical  in  an  informal  report  such  as 
this  to  list  all  these  publications,  particularly  since  many  of  the 
ideas  contributed  were  general  across  many  different  publications. 
However,  where  one  document  was  the  source  of  most  of  one  question  (or 
a  group  of  questions)  or  a  unique  definition  was  used  (i.e., 
structural  programming,  HIPO,  chief  programmer,  Orthodox  Job  Enrich¬ 
ment,  etc.)  a  reference  is  given.  We  hope  nobody  was  slighted. 
REFERENCES 

AFR  300-15,  Automated  Data  System  Project  Management,  USAF, 
Washington,  DC  (16  Jan  1978) 

ASPR,  Armed  Services  Procurement  Regulation,  Section  III,  Part  4, 
"Type  of  Contracts,"  Dept  of  Defense  (1976) 

Baker,  F.  T.,  "Chief  Programmer  Team  Management  of  Production 
Programming,"  IBM  System  Journal,  Vol  11,  Spring,  (1972)  pp  56-73 

Black,  Rachel  K.  E. ,  BCS  Software  Production  Data,  BCS  Report 
F30602-76-C-0174,  (Prepared  for  Air  Force  Rome  Air  Development 
Center),  Boeing  Computer  Services,  Inc.,  Seattle  (1977) 

Boeing  Computer  Services,  Inc.,  "Software  Quality  Assurance  Guide," 
Prepared  by  BCS/S/\MA  Division  Technical  Staff,  Document  Nr 
BCS  40111  (Jun  1976) 

Brooks,  F.  P.,  The  Mythical  Man-Month:  Essays  on  Software 
Engineering,"  Addison-Wesley ,  Reading,  MA  (1975) 

Computer  Decision,  Hayden  Publishing  Company,  Rochelle,  NJ 
(Jun  1977) 

Dijkstra,  E.  W. ,  Notes  on  Structure  Programming,  In  0.  J.  Dahl, 

E.  W.  Dijkstra,  and  C.  A.  R.  Hoare,  Structured  Programming, 

Academic  Press,  New  York  (1972)  pp  155-165 

DOD  Manual  4120. 17M,  Automated  Data  System  Documentation  Manual, 
(Dec  1972) 

Drurker,  Peter  F. ,  The  Practice  of  Management ,  Harper  and  Row, 

New  York  (1954) 
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Hartwick,  R.  Dean,  "Test  Planning,"  Proceedings,  1977  National  Computer 
Conference,  AFIPS  Press,  Montvale,  NJ  (1977) 

Herzberg,  Frederick  I.,  "Orthodox  Job  Enrichment:  A  Common  Sense 
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appendix  a 


CONTRIBUTORS 

INTRODUCTION 

This  appendix  lists  those  individuals  (usually  project  managers)  and 
firms  who  completed  the  survey-  This  list  is  provided  to:  1) 
acknowledge  the  contribution,  hard  work  and  willingness  to  contribute 
to  the  general  knowledge  of  computer  science  by  these  individuals;  and 
2)  to  lend  credibility  to  this  report  by  making  visible  the  excellent 
source  of  the  data. 

These  people  and  companies  are  all  members  and  supporters  of  the  AIAA 
Technical  Committee  on  Computer  Systems. 

At  the  end  of  this  list  is  a  group  of  individuals  that  wished  to 
remain  anonymous  in  order  that  they  could  provide  more  candid,  truthful 
answers . 


It  was  obvious  from  the  answers  received  that  the  contirubtors  worked 
very  hard  making  the  answers  as  truthful  as  possible.  Again,  the 
authors  thank  you. 


CONTRIBUTORS 

Mr  Philip  S.  Babel 
Technical  Advisor  for  Computer 
Systems  Acquisition 

Mr  Francis  J.  Barrett 
Chief,  PEACE  SIGMA  Development 
Unit 

Mr  Frank  L.  Bernstein 
Vice  President 
Federal  Systems  Division 

Mr  Herman  S.  Binder 

Section  Head,  Systems  Design, 

Analysis  &  Integration  Section 

Dr  M.  Lenard  Birns 
Program  Manager,  Naval 
Warfare  Gaming  System 

Mr  Jack  E.  Bloodworth 
Manager,  ALCM  Software 

Mr  David  A.  Brown 

Chief,  ARRCS  Development  Group 

Mr  Allen  G.  Burgess 
Manager,  Computer  Systems 
Laboratory 


Simulator  Systems  Program  Office 
Aeronautical  Systems  Division 
Wright-Patterson  AFB  OH  45433 

Data  Automation  Branch 
Sacramento  Air  Logistics  Center 
McClellan  AFB  CA  95652 

CALCULON  Corporation 
1501  Wilson  Boulevard 
Arlington  VA  22209 

Grumman  Aerospace  Corporation 
Bethpage  NY  11714 

Defense  Systems  Division 
Computer  Sciences  Corporation 
304  West  Route  38,  Box  N 
Moorestown  NJ  08057 

The  Boeing  Aerospace  Company 
P.J.  Box  3999 
Seattle  WA  98124 

Data  Automation  Branch 
Sacramento  Air  Logistics  Center 
McClellan  AFB  CA  95652 

Equipment  Division 
Raytheon  Company 
528  Boston  Post  Road 
Sudbury  MA  01776 
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Mr  George  R.  Cannon  Jr. 

Manager  of  Vandenberg  Programs 

Mr  Frank  J.  Cerulli 
Director  of  Engineering 
Computer  Systems  Division 
also 

Products  Systems  Division 
Mr  James  P.  Chilton 

Director,  Data  Processing  Sub  Systems 
Systems  Technology  Program 

Mr  Arthur  C.  Ciccolo 
Associate  Division  Leader 
Computer  Science  Division 

Mr  James  W.  Clark 

Manager  of  Engineering  Operations 

Mr  Jerry  E.  Cummings 
Program  Analyst 

Logistics  Research  &  Systems  Division 

Mr  G.  Russell  Curtis 

Manager,  Simulation  &  Data  Systems 

Information  Systems  Programs 

Mr  Alan  J.  Deerfield 
Consulting  Scientist 

Mr  Edward  M.  Dunaye 
Director,  Quality  Assurance 

Mr  Joe  N.  Dyer 

Manager ,  Equipment  Evaluation  & 
Systems  Prograimning 

Mr  Richard  R.  Erkeneff 
Chief  Design  Engineer,  Data 
Control  &  Processing  Systems 

Mr  S.  G.  Evetts 
Project  Manager 

Dr  George  R.  Fath 
Acting  Manager 

Avionics  Development  Engineering 


Logicon,  Incorporated 
P.O.  Box  1567 
Vandenberg  AFB  CA  93437 

Lockheed  Electronics  Company, 
Incorporated 
U . S .  Highway  22 
Plainfield  NJ  07061 

McDonnell  Douglas  Astronautics 
Company 

5301  Bolsa  Avenue 
Huntington  Beach  CA  92647 

The  Charles  Stark  Draper 
Laboratories,  Incorporated 
555  Technology  Square 
Cambridge  MA  02139 

United  Technologies  Research 
Center 

East  Hartford  CT  06108 

Directorate  of  Plans  &  Programs 
Sacramento  .4ir  Logistics  Center 
McClellan  AFB  CA  95652 

General  Electric  Company 
450  Persian  Drive 
Sunnyvale  CA  94086 

Submarine  Signal  Division 
Raytheon  Company 
P.O.  Box  360 
Portsmouth  RI  02871 

Planning  Research  Corporation 
7600  Old  Springhouse  Road 
McLean  VA  22101 

Lockheed  Missile  &  Space 
Company,  Incorporated 
P.O.  Box  504 
Sunnyvale  CA  94088 

McDonnell  Douglas  Astronautics 
Company 

5301  Bolsa  .4venue 
Huntington  Beach  CA  92647 

Vought  Corporation 
P.O.  Box  5907 
Dallas  TX  75222 

General  Electric  Company 
901  Broad  Street 
Utica  NY  13503 


Mr  Herb  Finnie 

Manager,  PLSS  Software  Development 


Mr  J.  I.  Freeman 

Avionics  Project  Engineering 

Dr  Virgil  "Smokey"  V.  Griffith 
Chief,  Electronics  Engineer 
Digital  Computer  &  Software 
Engineering 

Mr  Harvey  I.  Gold 

Manager,  Software  Technology  Dept 

Dr  Kenneth  A.  Hales 
Manager,  MSP  Mission  Control  & 
Software 

Mr  Uwe  W,  lbs 
Design  Specialist 

Dr  Peter  R.  Kurzhals 
Director,  Guidance,  Control  & 
Information  Systems  Division 

Mr  John  C.  Lemanczyk 
Manager,  Software  Technology 
Development 

Mr  Myron  Lipow 

Senior  Staff  Engineer,  Product 
Assurance 

Systems  Engineering  &  Integration 
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APPENDIX  3 
QUESTIONNAIRE 


INTRODUCTION 

This  appendix  contains  Part  Two  of  the  questionnaire,  plus  Questions 
1  through  4  and  25  of  Part  Three  (renumbered  Questions  160  through  164 
of  Part  Two) .  Other  reports  will  contain  the  balance  of  the  question¬ 
naire  . 

The  questionnaire,  as  printed  in  this  report  is  a  modified  version  of 
the  questionnaire  as  originally  answered.  This  was  done  for  the 
following  reasons; 

(1)  Not  all  questions  had  accompanying  multiple  choice  answers 
but  were  narrative  in  nature, 

(2)  The  original  questionnaire  contains  space  for  project  managers 
to  add  their  own  comments  as  answers  to  the  questions  rather  than  select 
one  of  Che  pre-given  answers,  and 

(3)  There  were  errors  (typo  and  otherwise)  in  Che  original  survey 
which  needed  correcting. 

The  procedures  used  to  report  on  those  questions  that  did  not  have 
preselected  answers  was  to  modify  the  original  questionnaire  to  make  it 
"look  like"  the  authors  had  preselected  these  possible  answers  and  the 
participants  had  checked  them.  In  truth,  Che  answer  set  was  derived 
from  Che  submitted  answers.  To  indicate  which  questions  were  originally 
narrative  in  form,  a  notation  in  parenthesis  following  Che  question  will 
indicate  "originally  narrative." 

In  addition,  the  original  questionnaire  contained  space  for  project 
managers  to  add  their  own  comments  as  answers  to  the  questions  rather 
chan  select  one  of  the  pre-given  answers.  This  was  encouraged  by  the 
authors  in  order  to  insure  Chat  the  answers  to  the  questionnaire  were  as 
accurate  as  possible  and  not  distorted  by  forcing  the  participant  to 
only  select  from  our  pre-conceived  answers.  Again,  to  provide  structure 
so  the  answers  can  be  encoded  in  a  computer  data  base  system,  the 
"comment"  answers  .vere  grouped  and  the  possible  "answer  set"  expanded  to 
include  these  answers.  To  indicate  these  additional  answers  the  word 
"added"  will  be  placed  in  parenthesis  at  the  end  of  the  question. 
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In  contrast.  Questions  33,  51,  32,  67,  68,  90,  115,  127,  159,  and 
164  were  left  in  narrative  form.  This  was  done  because  of  the  extreme 
.lumber  of  different  answers  that  were  submitted.  These  questions  were 
very  open-ended  questions  pertaining  to  "lessons  learned"  and  "research 
needed,"  and  were  extremely  varied  and  opinionated.  These  questions  are 
answered  by  encoding  the  variety  of  answers  and  entering  this  code  on 
the  tabulation  sheets. 

Other  modifications  were  made  to  the  original  questionnaire  where  the 
participants  indicated  the  question  was  poorly  worded  or  where  the 
participants  modified  the  original  question  by  the  insertion  of  a  word 
or  phrase.  The  auf’-ors  inserted  these  in  the  interest  of  making  this 
version  of  the  q''_stionnaire  more  complete.  These  additions  to  the 
original  questions  and/or  original  answers  are  indicated  by  placing  the 
added  portion  in  brackets  "[  ]"  and  placing  the  word  "added"  in  parenthe¬ 
sis  at  the  end  of  the  question  or  answer. 

Where  typographical  errors  were  made,  the  questionnaire  was  corrected. 
The  corrected  word  was  contained  in  brackets  "[  ]"  and  the  word  "corr" 
placed  in  parenthtesis  at  the  end  of  the  question  or  answer. 

The  authors  hope  that  the  above  explanations  do  not  appear  to  be  too 
complex.  They  were  done  purely  in  the  interest  of  conveying  the  maximum 
amount  of  information  to  the  reader  about  the  original  questions  and  the 
possibJ.e  answers  presented  to  the  respondent.  The  questionnaire  follows. 

After  the  questionnaire  was  sent  out,  it  was  discovered  that  there 
were  two  Questions  68.  This  was  corrected  by  having  the  second  Question 
68  renumbered  69f. 

References  were  added  where  they  were  needed  or  where  the  addition  of 
a  reference  would  make  the  question  clearer.  An  abbreviated  source  is 
contained  in  brackets  "[  ]"  and  the  complete  source  follows  this 
appendix. 
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RULES  AND  CONDITIONS  FOR  PARTICIPATING  IN  SURVEY 


It  is  important  that  no  company,  or  individual  suffer  any  loss 
of  proprietary  information  or  receive  unfavorable  publicity  as 
a  result  of  this  survey.  Each  individual  participating  in  the 
survey  has  our  full  assurance  that  the  data  he  provides  vill  be 
treated  in  accordance  with  the  above  principles.  In  order  to 
achieve  this  we  stipulate  the  following: 

1.  Unless  specifically  authorized,  the  names  of  participating 
firms,  or  individuals  will  not  be  listed  in  the  report  as 
contributors. 


2.  The  anonymity  of  the  company,  department,  individual,  and 
project  will  be  preserved  in  every  instance. 

3.  .Any  proprietary  or  company  confidential  information,  if 
so  identified  (by  writing  "CONF"  beside  the  question)  will  be 
protected  and  used  only  in  deriving  statistical  data. 

4.  The  individual  completing  the  questionnaire  can  omit  the 
answer  to  any  question  without  invalidating  the  questionnaire. 

.  Only  if  cleared  for  further  dissemination  will  raw  data 
(completed  forms)  be  made  available  to  the  participating  AIAA 
TC  members,  should  such  request  be  made,  to  assist  them  in 
research  work,  of  their  o^m.  Without  exception,  all  company, 
department,  project,  and  individual  names,  as  well  as  responses 
identified  as  "CONF"  will  be  systematically  deleted  prior  to 
release. 


6.  If  so  requested  by  the  submitter,  only  statistical  data 
will  be  derived  from  the  survey,  and  the  survey  form  destroyed 
upon  publication  of  the  final  report. 


Though  it  is  seen  as  providing  benefits  to  all  participants, 
including  the  U.S.  Air  Force,  this  survey  is  not  sponsored  by 
the  U.S.  Air  Force,  or  any  individual,  group,  committee,  or 
company,  and  does  not  imply  any  obligation  on  tliu  part  of  the 
participants.  It  is  being  accomplished  solely  to  provide  data 
to  be  presented  at  the  AIAA  Conference,  Computers  in  Aerospace, 
31  Oct  -  2  Nov  1977,  Los  Angeles,  California. 


ii 
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MEMO  OF  UNDERSTAND IN'G 


I  nuike  the  following  stimulations  under  which  this  survey  can 
be  used:  (Please  sign  each  stipulation  you  wish  to  make  as 
precondition  to  submitting  this  survey.  Line  through  those 
paragraphs  which  do  not  apply.) 

This  survey  with  company,  department,  project;  and  other 
identifying  markings,  and  with  all  answers  marked  "CONE" 
deleted  can  be  duplicated  and  provided  to  the  TC  members  at 
their  request. 


_  Signature  of  Submitter 

This  survey  can  only  be  used  to  provide  statistical  data  and 
cannot  be  released  to  the  TC  members  for  their  use  in  any  but 
a  composite  statistical  or  summary  form.  Following  publication 
of  final  report  both  this  form  and  the  survey  must  be  destroyed 
by  shredding,  pulping,  or  similar  means. 

_  Signature  of  Submitter 

I  authorize  the  release  of  the  firm  name  in  a  list  of  participants 
to  be  included  as  an  addendum  to  the  final  report.  The  desired 
name,  title,  etc.,  is: 


Signature  of  Submitter 


233 


A  SURVEY  OF  MANAGEMENT  TECHNIQUES  AND  PROCEDURES 
EMPLOYED  IN  SOFTWARE  DEVELOPMENT  PROJECTS 
INSTRUCTIONS 

Each  survey  packet  comprises  three  parts.  The  number  of  packets 
provided  will,  in  most  cases,  match  the  number  of  projects  to  be 
reported  on  plus  one  spare.  If  more  forms  are  required  you  may 
copy  or  call. 

PART  ONE  of  the  survey  deals  with  defining  the  total  organization 
and  the  overall  management  structure,  requirements,  and  philosophy, 
and  is  intended  to  be  answered  by  top  management .  It  provides  the 
backdrop  against  which  the  individual  projects  are  to  be  viewed. 
Normally,  only  one  copy  of  P.ART  ONE  should  be  completed  per  mailing, 
but  each  packet  contains  PARTS  ONE,  TWO,  and  THREE  for  the  sake  of 
uniformity  and  the  chance  that,  in  some  instances,  additional  P.ART 
ONES  would  be  called  for. 

.A  PART  TWO  is  to  be  completed  for  each  project  reported  on,  and  is 
intended  to  he  completed  by  the  project  manager.  (It  is  assumed  the 
project  is  completed  or  almost  completed)  If  those  methods  now  often 
referred  to  as  Modern  Programmer  Productivity  T'-'chniques  (cop  aown 
design,  structured  programming,  et  al.)  are  being  used  in  whole,  or 
in  part,  in  your  development  activities,  you  should  consider  selecting 
a  representative  sample  of  before  and  after  projects  in  completing 
the  Survey. 

PART  THREE  consists  of  general  questions  not  related  to  any  specific 
project,  and  is  also  intended  to  be  completed  by  a  pro  j  ec  t  manager . 

One  PART  THREE  is  included  in  each  packet  on  the  assumption  that  each 
project  will  be  reported  on  by  a  different  project  manager.  If  one 
manager  reports  on  more  than  one  project,  he  or  she  would  only 
complete  PART  THREE  one  time. 

The  dynamic  nature  and  infinite  diversity  of  the  entire  field  of  Data 
Processing  has  kept  the  jargon  from  becoming  universally  defined.  For 
this  reason  we  have  attempted  to  avoid  terms  that  might  have  more  than 
one  meaning.  If  questions  appear  vague  or  imprecise,  feel  free  to 
call  for  clarification.  Or,  if  you  prefer,  rewrite  tl'e  question  to 
ask  what  you  believe  the  point  to  have  been,  or  to  relate  it  to  your 
particular  environment. 

The  answers  provided  for  each  question  are  not  the  universal  set  of 
possible  responses,  so,  if  yon  believe  selecting  one  of  the  canned 
replies  would  be  misleading  please  select  "other"  or  "comment"  and 
explain.  If  more  space  is  required,  write  in  the  margins  indicating 
the  number  of  the  question  being  answered.  If  a  question  defies 
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answering  either  through  complexity,  non-relevance  to  your 
environment,  or  excessive  research  feel  free  to  leave  it  blank  or 
enter  an  appropriate  comment.  If  you  write  "CONF"  in  the  left  margin 
adjacent  to  any  question,  that  response  will  be  treated  as 
confidential/proprietary  data  as  described  under  ".Rules  and 
Conditions..",  attachment  1  to  the  basic  letter. 

If  possible,  avoid  direct  reference  to  specific  firms,  projects, 
and  people.  Each  set  of  questionnaires  has  been  pumbered  in  order 
that  we  might  keep  related  responses  together  and  facilitate 
accounting.  Base  numbers  have  been  selected  at  random  and  no 
algorithym  has  been  employed  that  would  facilitate  pairing  firms 
with  forms. 

We  very  much  appreciate  the  time  and  effort  you're  putting  into 
this.  Your  time,  effort,  and  candor  are  essential  to  the  success 
of  our  joint  effort. 

Please  return  the  completed  surveys  in  the  return  envelope  provided 
or  mail  to: 

Colonel  Richard  H.  Thayer 
SM-ALC/ACD 

McClellan  AFB,  CA  95652 
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A  SURVEY  OF  M^VNAGEMENT  TECHNIQUES  AND  PROCEDURES 
EMPLOYED  IN  SOFTWARE  DEVELOPMENT  PROJECTS 
PART  TWO  (Modified) 

SECTION  1  -  PROJECT  IDENTIFICATION 


INTRODUCTION.  P.\RT  TWO  of  chis  survey  perCaiiis  co  the  management  of  a 
specific,  individual,  software  development  project  and  is  intended  to  be 
answered  by  the  Project  Manager .  It  is  assumed  that  the  project  being 
reported  on  is  complete  or  almost  complete. 


THE  IDENTIFICATION  NUMBER  ASSIGNED  THIS  FORM  IS _ 

Please  return  completed  form  in  envelope  provided  or  mail  to: 

Colonel  Richard  H.  Thayer 
SM-ALC/.ACD 

McClellan  .AFB,  CA  95652 

1.  What  was/is  your  position  in  relation  to  the  project  you  are 
reporting  on?  (Originally  narrative) 

a.  Project  manager 

b.  Software  project  manager 

c.  Project  manager's  supervisor  (Program  manager) 

d.  Technical  director  or  Technical  advisor 

e.  Project  individual 

f.  Corporate  officer 

g.  Customer 

z.  Other: _ 

PROGR^YM  IDENTIFICATION 

2.  In  what  applications/ functional  area(s)  did  this  [software] 
project  fall?  (added) 

a.  Commercial/business,  such  as  inventory  control,  payroll, 
accounting  and  finance,  etc, 

b.  Data  acquisition/ retrieval 

c.  Scientific,  such  as  engineering  calculations,  data 

reductions,  etc. 

d.  Simulation  or  modeling  applications 

e.  Process  control  to  include  embedded  computer  systems 
[Manley,  1974] 

f.  Command  and  control  systems 

g.  Management  information  systems 


[  i 


A 
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h.  Communication  systems,  message  switching 

i.  Computer  systems,  such  as  software  monitors,  compilers, 
operating  systems  and  other  system.,  software 

z.  Others: _ 

3.  The  project  was; 

a.  A  new  software  development 

b.  A  continuation  of  a  previously  completed  software 
development 

c.  A  major  modification  of  an  existing  software  system 

z.  Other; _ 

4.  If  the  project  was  a  major  modification  did  it  involve: 

a.  Transferring  existing  software  to  a  different  computer 

b.  Rewriting  application  software 

c.  Writing  a  new  operating  system 

y.  Project  was  not  a  major  modification  to  an  existing 
system 

z.  Other: _ 

5.  The  software  product  was  designed  for: 

a.  Commercial  off-the-shelf  computer  hardware 

b.  Commercial  off-the-shelf  operating  system 

c.  Modified  commercial  off-the-shelf  computer  hardware 
(added) 

d.  Modified  commercial  off-the-shelf  operating  system 

e.  Special  purpose  computer  [hardware]  system  (added) 

f.  Special  purpose  operating  system 

z.  Other: _ 

COMMERCIAL  HARDWARE  IDENTIFICATION 

6.  If  the  software  system  was  being  developed  for  commercial 
[computer]  iiardware,  was  selection  and  delivery  of  the  commercial 
hardware  part  of  the  overall  project?  (added) 

a.  Yes 

b.  No 

c.  No,  furnished  by  user  (added) 

y.  System  was  not  developed  for  commercial  off-the-shelf 
computer  hardware 

z. 


[  ] 
[  1 

[  1 

[  ] 
[  1 

[  I 
i  ] 

I  ] 


i  ] 

[  1 
[  I 
[  ] 


[  ] 
[  1 
[  ] 


Comment : 


7.  If  of f-che-sheif  [computer]  hardvvare  was  used,  in  what  mode 
of  operation  did  the  production  system  run?  (added) 


a.  Batch 

b.  Remote  batch/remote  job  entry  terminal  (RJET) 

c.  Interactive  processing  [real  time]  (added) 

d.  Transaction  processing 

e.  Stand,  alone 

y.  System  was  not  developed  for  commercial,  off-the-shelf 
computer  hardware 

z.  Other: _ 

d.  If  the  target  (production)  computer  for  tliis  software 
capability  was  an  off-the-shelf  commercial  system: 

a.  Give  manufacturer,  make,  and  model _ 

b.  Give  operating  system  employed _ 

y.  Host  computer  was  not  a  commercial  computer 

z.  Comment _ 

9.  If  the  host  (development)  computer  for  this  software 
capability  was  an  off-the-shelf  commercial  system: 

a.  Give  manufacturer,  make  and  model _ 

0.  Give  operating  system  employed _ 

y.  Host  computer  was  not  a  commercial  computer 

z.  Comment: _ 

SPECIAL  PURPOSE  HARDWARE  IDENTIFICATION 

10.  If  the  software  system  was  developed  for  special  purpose  hard¬ 
ware,  was  the  hardware  development  part  of  the  overall  project? 

a.  Yes 

b.  No 

y.  System  was  not  developed  for  special  purpose  hardware 

z.  Comment: _ 

11.  If  the  software  system  was  developed  for  special  purpose 
hardware  what  type  was  tills? 

a.  Embedded  central  processor 

b.  Special  terminals  such  as  those  employed  in  command 
and  control  and  airborne  systems,  etc. 

c . 


I 

i 
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Special  systems  such  as  radar  sensors,  or  process 
control  devices,  trainers,  etc. 


y.  Sys^®'"  not  developed  for  special  purpose  hardware 

z.  Other: _ 

12.  If  a  host  (development)  commercial  hardware  system  was  used 
in  developing  this  special  purpose  capability: 

a.  Give  manufacturer,  make,  and  model _ 

b.  Give  operating  system  employed _ 

c.  System  was  developed  on  special  purpose  computer 

y.  System  was  not  developed  for  special  purpose  hardware 
(added) 

z.  Comment : _ 

CUSTOMER  IDENTIFICATION 

13.  The  customer  or  user  was:  [Taken  partly  from  Computer 
Decisions  Subscription  Application  form,  Jun  1977  Issue] 

a.  In-house  to  my  company  or  major  division 

b.  Within  my  parent  organization 

c.  A  manufacturer  of  computer  hardware 

d.  A  manufacturer  of  other  than  computer  hardware 

e.  A  "software  house" 

f.  An  engineering  service  and  technical  support  organization 

g.  The  Government:  federal  (non-military),  federal 
(.military)  ,  state,  cou.ity,  municipal 

h.  A  university  or  educational  institution 

i.  A  computer  service  bureau,  time-sharing  service  bureau 

j .  An  ADP  consultant  and/or  education  service 

k.  Financial:  banking,  insurance,  real  estate,  securities, 
credit 

l.  In  the  wholesale  or  retail  trade 

m.  In  medical  or  legal  services 

n.  In  transportation  services 

o.  Utilities:  communications,  electric,  gas 

p.  Foreign  government  (added) 

z.  Other: _ 

14.  Was  the  customer  purchasing  the  software  system  for  other 
users? 

a.  Yes 

b.  No 


z. 


Comment : 
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15.  Are  (is)  there ; 

a.  Multiple  users  of  the  system 

b.  Only  one  user  [one  or  more  locations]  (adOed) 

z.  Comment  _ _ _ 

CONTRACT  IDENTIFICATION 

16.  Under  which  of  the  following  contract  types  or  agreements  was 
the  software  system  developed?  [.AFPR,  1976] 


a. 

Firm  fi.xed  price 

[ 

b. 

Fi.xed  price  with  economic  price  adjustment 

r 

i 

c . 

Fixed  price  incentive 

r 

d. 

Firm  fixed  price  level  of  effort 

[ 

e. 

Cost 

[ 

£. 

Cost  sharing 

[ 

g. 

Cost  plus  incentive  fee 

i 

h. 

Cost  plus  award  fee 

[ 

i. 

Cost  plus  fixed  fee 

[ 

j  • 

Time  and  materials 

r 

k 

k. 

Labor-hour 

( 

1. 

Basic  ordering  agreement 

1 

y- 

No  contract  was  used  (added) 

[ 

z . 

Other ; 

[ 

17.  If  the  contract  had  an  incentive  clause  [for  software],  what 
was  the  incentive  based  on?  (added) 

a.  Reduced  [or  meets]  cost  (added)  [ 

b.  Early  [or  meets]  delivery  (added)  [ 

c.  Increased  performance  (explain  how  it  was  measured) _  [ 

y.  Contract  did  not  have  an  incentive  clause  [ 

z.  Other: _ [ 

COST  AND  SCHEDULE  IDENTIFICATION 

18.  What  was  the  total  cost  of  the  project  to  the  customer?  (If 
exact  amount  not  available,  please  give  a  range  or  explain) 

$  _ 


19.  How  much  of  Che  coCal  cose  (escimaced)  was  actribucable  to 
Che  production  of  software,  including  machine  time,  programmer 
salary,  [management  of  software  development,  overhead  related  to 
software],  and  ocher  ADP  operating  expenses?  (added) 

i _ 

20.  This  project  began  (first  assignment)  in _ 

(month,  year)  and  ended,  [or  will  end]  (sign  off)  in _ 

(month,  year) .  (added) 

SOFTWARE  IDENTIFICATION 

21.  In  what  language  was  this  data  system  programmed?  If  more 
than  one  language  was  used  please  indicate  appropriate  percent¬ 
ages  . 

a .  F0RTR,\N 

b.  JOVIrU. 

c.  COBOL 

d.  Assembler  [(unspecified)]  (added) 

e.  CMS- 2  (added) 

f.  PL/i  (added) 

g.  Higher  order  language  [(unspecified)]  (added) 

z.  Other:  _ _ 

22.  In  total,  approximately  how  many  lines  of  source  code  were 
written? 

a.  Executable _ 

b.  Comments : _ 

c.  Ocher  non-executable _ 

COMPLEXITY  IDENTIFICATION 


23.  Using  Table  1  on  che  following  page  please  estimate  tiie 

complexity  of  che  data  system  (from  12  to  60).  All  descriptions 
pertain  to  software  only.  The  complexity  racing; _ 

DATA  BASE  IDENTIFICATION 

24.  Approximately  how  large  a  data  base  was  the  system  designed 

for:  (Fill  in  blank  with  number  of  bytes  of  data) _ 
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SECTION  2  -  REQUIREMENT  SPECIFICATIONS, 
INPUT  CONDITIONS,  AND  ENVIRONMENT 


REQUIREMENT  SPECIFICATIONS 


25.  Who  prepared  the  requirement  specifications? 

a.  The  customer 

b.  An  organization  affiliated  with  the  customer,  but  not 
the  customer  himself 


c.  Your  organization,  e.g.,  a  two-step  procurement,  an 
unsolicited  proposal,  etc. 

d.  An  outside  consulting  firm 

e.  A  third  contractor,  e.g.,  an  integrating  contractor 

y.  None  prepared  (added) 

z.  Other: _ 

26.  If  your  organization  prepared  the  requirement  specifications 
did  the  customer  participate? 

a.  Yes 


b.  No  [customer  did  not  participate]  (added) 

c.  No,  customer  was  source  (added) 

y.  Not  applicable,  your  organization  did  not  prepare 
specifications  (added) 

z.  Comment: _ 

27,  On  a  scale  of  1  to  7,  with  1  being  little  more  than  the  name 
of  the  system,  and  7  being  complete  specifications  dovvTi  through 
preliminary  design,  how  detailed  were  the  specifications  provided? 


I  ] 
[  ] 


[  1 


[  1 


28.  Was  it  necessary  to  rewrite  the  specifications  before 
proceeding  with  design? 

a.  Yes  (specify  percent  rewritten) _ ^  [  ] 

b.  No  [specifications  were  adequate]  (added)  [  1 

c.  Yes,  however  specifications  never  rewritten  (added)  [  j 

d.  Yes,  rewritten  concurrent  with  design  (added)  [  | 

y.  Not  applicable,  requirement  specifications  were  never  [  ] 

written  (added) 

z.  Comment: _ _  [  1 

29.  If  it  was  necessary  to  rewrite  the  specifications,  what  was 
the  reason?  (originally  narrative) 

a.  Errors  in  specification  [  1 
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30. 


b.  Specifications  were  ambiguous,  incompieto,  anci/or 
inconsistent 

c.  Change  in  scope  of  project 

d.  Change  in  requirements  of  project 

e.  Change  in  hardware  design 

£.  Normal  way  of  doing  business 

g.  Customer  and/or  developer  became  better  informed 

y.  Was  not  necessary  to  rewrite  specif ications  or 
requirement  specifications  were  not  prepared 

z.  Other; _ 

The  specifications  were  prepared: 

a.  Using  top-down  (Hierarchy  of  Function)  techniques 

b.  Using  structured  design/fiow  techniques 

c.  In  phases,  with  design  and  coding  started  before  the 
requirement  specification  was  completed 

d.  In  a  formal  requirement  language  (describe) _ 


e.  In  such  a  manner  as  to  facilitate  tracking  tiie  develop¬ 
ment  of  software  from  requirements  througii  coding  and 
tie  each  delivered  software  module  to  some  part  of  the 

requirements  (approximately _ 1  of  tiie 

requirement  could  be  cracked) . 

f.  In  precise,  measureable  terms  to  aid  in  development  of 

acceptance  testing  (approximately _ a  of  cne 

requirements  could  be  measured) . 

g.  According  to  MIL-STDs  (added) 

y.  Specification  never  prepared/none  (added) 

z.  Comment: _ 

DOCU'MENTATION  REQUIRED  3Y  THE  C0NTtl,\CT/ CUSTOMER 

31.  The  following  is  a  list  of  documentation  chat  may  be  required 
by  a  software  development  contract.  Check  each  document  required 
in  Che  project  and  add  required  documentation  not  contained  in  the 
list.  [DOD  Manual  4120. i7M,  (Dec  1972)) 

a.  Object  Liscing/Deck/Tape 

b.  Source  Listing/Deck/Tape 

c.  Functional  description — initial  definitioti  of  a  project 
which  provides  the  ultimate  user  with  a  clear  statement 
of  Che  operational  capabil  iti^.  s  to  bo  developed  (also 
sometimes  called  requirements  specifications) . 
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d.  Data  requirements  document — prepared  by  both  system  anu 
user  personnel  when  a  data  collection  effort  by  the 
user  group  is  required  to  generate  and  maintain  system 
files. 

e.  System/subsystem  specification — prepared  for  system 
personnel,  as  detailed  as  possible,  concerning  the 
environment  and  design  elements,  in  order  to  provide 
maximum  guidance  to  the  program  design  effort;  also 
defines  system  subsystem  interfaces. 

f.  Program  specifications — prepared  for  programmers; 
contains  considerable  detail  for  the  purpose  of  guiding 
program  development. 

g.  Data  base  specification — prepared  for  programmers  in 
sufficient  detail  to  permit  program  coding  and  data 
base  generation  by  the  programming  group. 

h.  Users  Manual — to  present  general  and  specific  information 
on  how  a  specific  computer  program  will  be  used  [(includes 
positional  handbook  for  remote  input  terminals) ] .  (added) 

i.  Computer  Operations  Manual — contains  precise  detailed 
information  on  the  control,  requirements  and  operating 
procedures  necessary  to  successfully  initiate,  run,  and 
terminate  Che  subject  system. 

j.  Program  Maintenance  Manual — contains  general  and 
specific  information  for  computer  personnel  responsible 
for  Che  maintenance  of  Che  computer  programs. 

k.  Test  and  Implementation  Plan — contains  an  orderly  schedule 
of  events  and  lists  of  material  necessary  to  affect 
testing  and  delivery  of  a  complete  data  processing  system. 

l.  Test  Analyst  Report — describes  the  status  of  the 
computer  program  system  after  test  and  provides  a 
presentation  of  deficiencies  for  review  by  staff  and 
management  personnel. 

m.  Software  development  plan  (added) 

n.  Software  standards  and  requirements  (added) 

o.  Management  reports  (added) 

p.  Interface  control  (added) 

q.  Budget  (added) 

r.  Progress  Reports  (added) 

s.  Error/discrepancy  report  (added) 

t.  Program  change  requests/staCus  (added) 

u.  Version  description  (added) 

Software  product/output  specifications  (added) 


V. 
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y.  Documencac ion  not  ic^aired  (.added) 

z.  Other; _ 

SPECIFIC  CUSTOMER  REQUIREMENTS 

32.  which  of  Che  following  were  specified  by  the  custoner? 


a.  A  specif.'c  i.ompucer  (this  was _ ) 

b.  Storage  limitations  (these  limitations  were _  ) 

c.  Speed  constraints  (e;<piain _ ) 

d.  A  specific  language  (this  language  was _ ) 

e.  His  participation  in  the  design  function 

f.  His  participation  in  the  coding  function 

g.  Prioritized  requirements 


h.  Development  under  life-cycle  costing  concepts 

i.  Development  under  design-co-cost  concepts 

j .  Development  under  modern  programming  techniques 

k.  Portability 

l.  Human  engineering 

m.  Security 

n.  Safety 

0.  MIS-S-52779 

p.  Modified  MIL-S-52799  (liow  modified) _ 

q.  Types  of  review 

r.  Frequency  of  review 

s.  Input  data 

C.  Output  requirements 

u.  Test  plan/procedure 

V.  A  particular  reliability  figure  to  attain  (if  so,  what 
method  of  measuring  software  reliability  did  the 
customer  specify?) _ 

w.  A  maintainability  goal  to  attain  (if  .so,  what  method  of 
measuring  software  maintainability  did  the  customer 
specify?) _ 

y.  Customer  did  not  specify  -(added) 

z.  Comment:  _ 


246 

aa.  A  followup  maincenance  coatracc  (time  in  :aonc.iis _ 

bb.  Customer  training 

cc.  A  warranty  of  the  software  for  a  period  of  time  (give 
time  in  calendar  months) _ 

GENERAL 


33.  If  you  could  affect  the  method  by  which  requirements  were 
specified,  or  were  able  to  initiate  research  into  improving  the 
requirements  specification  function,  wiiat  action  would  you  take 
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SECTION  3  -  PUiNNlNG 

PLANNING  AND  SCHEDULING 

34.  Who  did  Che  planning  before  Che  projecC  was  assigned  Co  :i 

group  or  individual  for  accompiishmenC? 

a.  A  [permanenc]  planning  group  sec  up  for  chis  purpose 
(added) 

b.  An  ad  hoc  planning  group  (added) 

c.  A  sceering  commiccee  which  escablished  coses  and 
schedules 

d.  An  on-going  projecc  (added) 

a.  Program  manager  was  picked,  and  he  did  che  inicial 
planning 

f.  A  scaff  funecion 

y.  No  planning  accomplished  (added) 

z.  Ocher: _ 

35.  In  planning  for  chis  projecc  did/was: 

a.  The  projecc  manager  use  a  formal  planning  guide? 

b.  The  cuscomer  acCively  parcicipace  in  che  planning 
funecion? 

c.  A  work  breakdown  scruccure  (WBS)  employed  in  planning 

che  sofeware  developmenc?  (approximaceiy  how  many 
levels) _ 

d.  The  projecc  divided  into  separate  phases  for  che  purpose 
of  planning? 

y.  None  (added) 

z.  Comment: _ 

36.  What  tools  were  used  in  planning  che  projecc? 


a. 

PERT 

[ 

] 

f . 

Milestone  (added) 

b. 

Modified  PERT 

[ 

1 

g- 

CSCS  (added) 

c . 

[CPM]  (corr) 

[ 

1 

h. 

WBS  (added) 

d. 

GANTT 

[ 

1 

y- 

None 

e. 

Workloading  cliarcs 

[ 

1 

z . 

ocher : 

What 

method  was  used  in 

escimat ing 

che 

cose  and  schedule  for 

che  projecc? 

a.  A  formula  [top-down)  (added) 


248 


b.  EsCimaCes  based  on  a  similar  project  J 

c.  Provided  by  somebody  who  iiad  a  knack  for  estimating  [  j 

correctly 

d.  Bottom-up  aggregating  (added)  [  ] 

e.  Cost  and/or  schedule  were  dictated  (added)  [  ] 

f.  Simulation  model  [  ] 

g.  Crystal  ball  (or  similar  means)  [  ] 

y.  Cost  and  schedule  not  estimated  (added)  [  ] 

z.  Other: _ [  ] 

33.  IE  the  formula  approacli  [and/or  other  estimating  iechniquei 
was  used  what  were  the  elements  or  variables  considered?  (added) 


[Olsen,  1977] 

a.  Computer  time  [  ] 

b.  Documentation  [  ] 

c.  Training  [  ] 

d.  Travel  [  ] 

e.  Site  preparation/  [  ] 

construction 

f.  Ratio  manage-  [  ] 

ment  4  overhead 

to  programmers 

g.  Keypunch  [  ] 

h.  Office  supplies  [  ] 

i.  Office  space  [  ] 

j.  Personnel  equip-  [  ] 

ment ,  i. e. ,  desks , 
pencils,  paper,  etc. 

k.  Programmer  profic-  [  ] 

iency 

l.  Program  comple.xity/  [  ] 

function 

m.  Tester  proficiency  [  ] 


n.  Number  of  modules  (sub  [  ] 

routines) 

o.  Lines  of  code  (adued)  [  ] 

p.  Size  of  program  [  ] 

(added) 

q.  Requirement  specific-  [  ] 

action  completeness  (added) 

r.  Customer  support  [  j 

(added) 

s.  Historical  data  (added)  [  ] 

t.  Availability  of  [■] 

support  software  (added) 

u.  Calendar  period  (added)  [  ] 

V.  Size  of  data  base  [  ] 

(added) 

w.  E;<perience  (newness)  [  ] 

with  system  (added) 

y.  Not  used  [  j 

z.  Other: _ [  ] 
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i9.  Afcer  Chti  original  soiiwar<i  dovciopmenc.  plan  and  scheduia  were 
decerTTiined  and  submitted  for  review  by  senior  management  and/or 


customer 

was : 

a. 

The 

delivery 

data 

shortened  by  senior  management 

[ 

b. 

The 

delivery 

data 

shortened  by  the  customer 

L 

c. 

The 

delivery 

data 

lengthened  by  senior  management 

L 

1  d. 

1 

The 

delivery 

date 

lengthened  by  the  customer 

r 

i 

'  e. 

The 

deli  very 

data 

originally  dictated  (added) 

[ 

1 

The 

requirements  c 

hanged  to  match  sciiedule  (added) 

[ 

g- 

Resources  reduced 

(added) 

L 

y- 

No 

change  (added) 

Z  . 

Comment : 

r 

40.  In  the  process  of  software  system  development  which  of  the 
following  planning  documents  were  prepared? 


a. 

Project  management 

[ 

)  k. 

Data  conversioii 

b. 

Resource  require¬ 

[ 

1  1. 

Change  control 

ments 

m. 

Configuration  manage¬ 

c. 

Organization 

( 

] 

ment  (added) 

d. 

Staf  f ing 

( 

1  11. 

Review  i  reporting 

e. 

Training 

[ 

)  0. 

Product  assurance 
(added) 

f . 

Test 

[ 

] 

P. 

Work  Breakdown  Structure 

S- 

Software  develop¬ 

1 

] 

(added ) 

ment 

q- 

Budget  (added) 

h. 

Phase  and/or 

[ 

] 

delivery 

y. 

None 

i. 

Documentation 

[ 

1  z. 

Other : 

j  • 

Implementation 

[ 

1 

From 

project  inception 

through  delivery 

of  the  completed 

system,  approximately  what  percent  of  the  time  was  spent  by  all 
personnel  in  planning  for  software? 

42.  In  a  furtlier  breakdown  of  planning  activities,  aijprox imately 
wiiat  percentage  of  the  time  was  spent  on  each  of  the  following? 
(Should  approximate  100/i) 

a.  Organizational  planning 

b.  Staff  planning 

c.  Developing  Control  procedures 

1 


250 


d. 

Administration  planning 

e. 

Quality  assurance  planning 

f. 

Developing  an  overall  project  management  plan 

g- 

Design  standards  (added) 

h. 

Test  planning  (added) 

i. 

Document  and  configuration  management  planning 
(added) 

/a 

y. 

.None  (added) 

[  1 

Z  . 

Other : 

[  1 

43.  The 

system  was: 

a. 

Delivered  to  the  user  as  an  entity 

r  ’ 

L  J 

b. 

Delivered  to  the  user  in  phased  increments 

r  ’ 

L  i 

z. 

Other ; 

_  [  1 

44.  Was  every  major  software  module  designed  in  total  before  any 


coding  was  started? 


a. 

Yes 

b. 

No  (percent 

of  design  completed  wiien  coding 

started  _ 

%) 

z . 

COMMENT : 

QUALITY  ASSURANCE  (QA)  PROGFOiM  PL.\N 


45.  A  QA 

program  was: 

a. 

Applied  informally 

f 

1 

b. 

Applied  formally  and  documented 

separately 

[ 

c. 

Applied  formally  and  documented 
management  plan 

as  part  of  the  project 

[ 

y- 

Not  applicable  to  [or  used  on] 

this  project  (added) 

[ 

z. 

Comment : 

[ 

46.  In  which  of  the  following  areas  were  formal  [and/or] 
documented  QA  standards  applied  and  were  these  company-wide 
standards  or  standards  developed  specifically  for  this  project? 

(added)  [Boeing,  1976] 

A  li 

Company-Wide  Local  to 

Standards  Pro  j  ect 


a. 

Documentation 

[  ] 

[  ] 

b. 

Work  breakdown  structure  (WBS)  code 

[  ] 

[  ] 

c. 

Scheduling 

[  1 

[  ] 
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A 

Company-Wide  Loc 


S  candards 

Pro 

d. 

Performance  measurement 

[  ] 

f 

e. 

Requirement  analysis 

[  ] 

L 

f . 

Preliminary  design 

[  ] 

[ 

s- 

Detail  design 

[  1 

[ 

h. 

Coding 

[  1 

r 

1 

i- 

Test  planning 

[  1 

f 

j  • 

Software  verification 

[  1 

[ 

k. 

Reviews  and  audits 

[  1 

[ 

1. 

Configuration  management 

[  1 

[ 

m. 

Discrepancy  reporting  and  correction  [  ] 

\ 

1 

n. 

Software  acceptance 

[  1 

f 

i 

y. 

No  formal  [or  documented]  iJA 
standards  (added) 

[  1 

I 

z . 

Other ; 

[  ] 

1 

47.  Which  of  Che  "modern  programmer  practices"  given  below  were 
used  in  the  software  development  project? 

a.  Program  manager  authority — both  technical  and  [  1 

administrative  responsibility  for  project  [black,  1977] 

b.  Reviews — formal  milestone  reviews  at  the  end  of  eacii  [  ] 

phase  f ftlack,  1977) 

c.  Unit  development  fo.'.ders — capture  of  working  materials  [  ] 

for  each  identified  item  to  facilitate  end  item  develop¬ 
ment,  testing,  documentation  [Ingrassia,  1976] 

d.  Design  discipline  and  verification — top-down  design,  [  ] 

formal  design  representation,  completion  of  design, 

and  deliberate  verification  of  design  prior  to  code 
[Black,  1977] 

e.  Program  modularity — dcf initions/restrictions  on  data  [  ] 

interfaces  between  modules,  adlierence  to  parent/ciiild 
relationships  between  modules.  [Black,  '9771 

f.  Naming  conventions — structured  names  for  modules  .uid/or  i  | 

data  items  [Black,  1977] 

g.  Structured  forms — use  of  Dijkstra  forms  as  supportable  [  ] 

in  your  programming  language  [1972] 

h.  Structured  walkthroughs — deliberate  peer  reviews  of  [  ] 

code  for  each  module  [Weinberg,  1971] 


rol  r) 


252 


i.  Scruccured  analysis — a  foroiai  descripcion  ol  the  usars 
requirements  [Yourdon,  1976] 

j.  Structured  design — building  system  from  small,  liighly 
independent,  single  purpose  modules  [Yourdon,  1976] 

k.  Chief  programmer  teams — a  development  organization 
around  a  programmer  of  great  ability  [Baker,  1972] 

l.  HIPO — a  documentation  technique  [IBM,  1975] 

m.  Support  libraries  and  facilities — use  of  automated  or 
proceduralized  design,  coding  and  configuration  manage¬ 
ment  acceptance  testing  [Black,  1977] 

n.  Phased  testing — defined  and  formalized  unit,  functional, 
and  acceptance  testing.  [Black,  1977] 

0.  Configuration  management/change  control — creation  and 
control  of  baselines  and  procedures  for  problem 
reporting  and  resolution.  [Black,  1977] 

y.  None  of  the  above  (added) 

z.  Other: _ 

48.  Which  of  the  below  listed  documentation  types  wore  used  in  the 
project?  If  a  single  document  covered  more  than  one  area,  check 
each  area  accounted  for.  The  intent  of  this  question  i;;  to 
determine  which  areas  were  covered  by  documentation  not  the  actual 
document  form.  [See  Section  2  for  definitions] 

a.  Functional  description 

b.  Data  requirements  document 

c.  System/subsystem  specification 

d.  Program  specifications 

e.  Data  base  specifications 

f.  Users  manual 

g.  Computer  operations  manual 

h.  Program  maintenance  manual 

i.  Test  analyst  report 

j.  Test  plans  (added) 

k.  Interface  control  (added) 

l.  Training  course  (added) 

m.  Version  description  (added) 

n.  Software  development  plan  (added) 

y.  None  (added) 

z.  Ocher: 


L  1 


i  ] 

[  1 
[  1 


[  ] 
[  1 
[  ] 
[  1 


[  ] 
[  ] 
(  ] 


[  ] 
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49.  Which  software  toois/aids  were  selected  for  use  on  tiiis 
project? 

a.  Structured  pre-compilers 

b.  Automatic  flowcharters 

c.  Library  monitors 

d.  Macro  programming  capabilities 

e.  On-line  capabilities 

f.  On-line  debugging 

g.  Automatic  test  case  generators 

h.  Simulators  or  drivers  (added) 

i.  Batch  debugging  aids  (added) 

j.  Standards  auditors  (added) 

y.  No  software  tools/aids  employed 

2.  Other: _ _ 

50.  Which  test  tools/ techniques/methods  were  selected  for  use  on 
this  project?  [Hartwick,  1977] 

a.  Comparators — used  to  compare  the  code  of  one  program 
version  to  that  of  another  for  coding  errors 

b.  Editors — analyze  source  code 

c.  Flowcharting— show  the  logic  structure  of  a  program 

d.  Logic/Equation  generators — translates  assembly  language 
programs  into  a  machine-independent  microprogramming 
language  and  builds  the  microprogramming  statements 
into  networks  to  analyze  the  flow  of  control  and 
reconstruct  arithmetic  statements 

e.  Program  structure  analyzers — analyze  program  paths 
under  input  control 

f.  Correctness  Proofs — establishes  that  a  given  program 
performs  a  desired  function  and  halts 

g.  Symbolic  Program  Executions — decompose  source  code  by 
logically  executing  it 

h.  Initialization  Tests — the  performance  of  all 
initialization  operations  is  tested  to  assure  that  all. 
indicated  initialization  operations  are  performed  and 
that  correct  values  of  all  initialized  quantities  result 

i.  Interaction  Test — all  quantities,  variables,  and  system 
conditions  obtained  from  other  modules  are  examined  to 
determine  the  sensitivity  of  the  module  under  test  to 
their  possible  values  or  states 


I 
[ 

[ 

[  I 

[ 

[ 

! 

I 

i 

I 

I  1 
L  i 


[  j 
[  1 
[  I 


[  J 
[  i 
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j.  Arichmetic  Test — Che  precision  of  arithmetic  calculations  [  ] 
is  checked  to  discover  where  insufficient  precision  is 
attained  or  incorrect  arithmetic  calculations  arc 

performed 

k.  Timing  Analysis — the  longest  and  shortest  possible  [  ] 

execution  times  for  all  tests  are  determined  to 

establish  the  execution  time  requirements  for  the  module 
and  to  identify  potential  timing  problems 

l.  Branch  Logic  Test — the  correct  branching  decision  paths  [  J 

for  each  branch  and  each  closed-loop  test  case  are 

checked.  The  branch  decision  paths  that  arc  not 
exercised  in  any  of  the  normal  test  cases  are  identified 
and  their  correctness  demonstrated  by  special  test  cases 

m.  Comparison  of  system  output  with  manual  output  tiie  system  [  j 

will  replace  (added) 

n.  Real  world  simulation  (added)  [  ] 

y.  None  (added)  [  J 

z.  Other: _  [  ) 

GENERAL 

51.  If  you  had  total  control  of  the  planning  .function  within  your 
organization,  or  were  able  to  initiate  research  into  how  to  improve 
the  planning  function,  what  actions  would  you  take? 


52.  If  it  were  in  your  power  to  make  changes  in  the  way  technical 
decisions  are  made  concerning  programming  techniques,  test  procedures, 
documentation  standards,  etc.,  what  action  would  you  take? 
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SECTION  4  -  ORGAMZATION 
DEFINITIONS  [Heyel,  1973] 

Line  Organizacion — A  line  organization  has  a  direct  line  of  respons¬ 
ibility  and  control  from  the  chief  executive  or  general  manager  to  inter¬ 
mediate  line  executives,  to  foreman  and  supervisors,  to  workers.  A  line 
or  operating  organizational  unit  is  one  that  is  actually  do ing  t:ie  work 
that  represents  the  primary  mission  of  the  larger  organ Lzat ionai  unit. 
Examples  are  aircraft  manufacturing,  radar  construction  and  installation, 
supply  and  transportation  of  spare  parts,  building  and  launching  a  MM\S 
lander,  teaching  in  a  university,  etc. 

Staff  Organization — A  staff  organization  is  given  respons io ii  i  ty  and 
autliority  over  special  activities,  suc!i  as  inspection,  employment, 
purciiasing,  legal,  engineering,  automatic  data  processing  (ADP)  ,  etc. 
staff  or  service  organizational  unit  is  any  unit  wiiicii  is  helping  the 
line  do  its  work,  but  is  only  responsible  for  the  special  activity  not 
the  final  product  of  the  larger  organizational  unit. 

Matrix  Organization — A  matrix  organization  is  built  around  specific 
projects.  A  manager  is  given  the  authority,  responsibility,  and 
accountability  for  completion  of  the  project.  The  line  or  staff  organ¬ 
izations  provide  qualified  personnel  when  needed,  who  return  to  tneir 
parent  organization  when  their  cask  is  done.  The  project  manager  usually 
does  not  have  authority  or  responsibility  to  hire,  disciiarge,  train,  or 
promote  personnel.  This  is  the  responsibility  of  the  line  or  staff 
manager . 

Project  Organization — A  project  organization  is  similar  to  a  matrix 
organization  except  that  Che  personnel  are  permanently  assigned  to  die 
organization.  The  manager  is  also  given  the  authority,  responsibility 
and  accountability  for  completion  of  Ciie  project.  However,  the  manager 
must  meet  his  goals  within  the  resources  of  his  organizacion.  T'ae 
manager  usually  has  responsibility  to  hire,  discharge,  train,  and  promote 
personnel  within  his  project  organization. 

PREORGANIZATIONAL  FUNCTION 


53.  To  which  organizacion  within  the  firm  was  this  software 
development  project  first  assigned: 

a.  A  line  organization  under  the  authority  of  the  senior  |  | 

ADP  manager 

b.  A  line  organizacion  outside  the  authoriCy  of  the  senior  (  1 

ADP  manager  [or  no  senior  ADP  manager]  (added) 

c.  A  staff  organizacion  under  the  autliority  of  Che  senior  [  ] 

ADP  manager 

d.  A  staff  organization  outside  die  authority  of  die  senior  [  j 

ADP  manager  [or  no  senior  ADP  manager]  (added) 


z. 


Ocher : 
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5“+.  Which  organizacion  was  responsible  for  determining  die 
initial  budget,  delivery  schedule,  resource  requirements,  computer 
availability,  etc.,  [for  software]?  (added) 

a.  The  organizacion  to  which  the  project  was  initially  [  ] 

assigned 

b.  A  line  organizacion  under  Che  authority  of  the  senior  [  ] 

ADP  manager 

c.  A  line  organizacion  outside  Che  authority  of  the  senior  [  ] 

ADP  manager  [or  no  senior  ADP  manager]  (added) 

d.  A  staff  organizacion  under  the  authority  of  the  senior  [  ] 

ADP  manager 

e.  A  staff  organizacion  outside  the  authority  of  ciie  senior  [  j 

iVDP  manager  [or  no  senior  ADP  manager]  (added) 

z.  Ocher; _  [  | 

When  this  project  was  initially  assigned: 

a.  The  ultimate  proje  ;  manager  participates  in  determining  [  | 

Che  schedule,  budget  figures,  etc. 

b.  The  planning,  budgeting,  and  allocation  of  resources  was  [  ] 

done  by  a  special  staff  established  for  this  function. 

(give  name _ ) 

c.  An  ad  hoc  group  was  set  up  to  handle  Che  initial  [  ] 

assignment  until  a  permanent  group  was  established 

z.  Comment: _  [  ] 

PROJECT  M^iNAGEMENT  ORGANIZATION 

56.  To  which  organizacion  was  the  [software]  project  assigned  for 


develo,yi 

.lent?  (added) 

a. 

The  organization  to  which  Che  project  was  initially 
assigned 

[ 

b. 

The  organizacion  responsible  for  determining  initial 
budget,  schedule,  etc. 

[ 

c. 

A  line  organizacion  under  the  authority  of  Che  senior 
<\DP  manager 

[ 

d. 

A  line  organizacion  outside  Che  authority  of  the  senior 

ADP  manager  [or  no  senior  ADP  manager]  (added) 

[ 

e. 

A  staff  organizacion  under  the  authority  of  the  senior 

ADP  manager 

[ 

f . 

A  staff  organizacion  outside  the  authority  of  Che  senior 
ADP  manager  [or  no  senior  ADP  manager]  (added) 

[ 

z. 


Ocher : 
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57.  Tlie  software  development  was  organized  under  [the  loilowirii; 
project  types):  (added) 

a.  A  project  organization 

b.  .A  matrix  organization 

c.  A  project  manager  with  administrative  authority  wi\ile 

the  actual  development  work  was  done  by  line  and  staff 
organizations 

y.  Not  organized  under  a  project  [type]  organization  (added) 

58.  In  this  project: 

a.  Software  development  was  handled  witain  tiie  AhP 
environment  with  functioiiiil  an.ilysts  or  prosi'cc  t  ive  users 
being  assigned  or  attached  to  the  development  team 

b.  ADP  specialists  were  detailed  or  assigned  to  c;\e 
functional  user  for  the  duration  of  tiie  development 
effort 

c.  Tlie  functional  user  employed  "analysts"  who  developed 
specifications,  designs,  algoritlims,  etc.,  whicli  were 
then  presented  to  the  software  development  team  for 
implementation 

d.  The  j\DP  function  was  done  by  the  functional  analyst/user 
(i.e.,  ADP  was  not  a  separate  function)  (added) 

e.  The  functional  analysis  was  done  by  tiie  ADP  ;versoniieI 
(i.e.,  there  was  not  a  separate  user  .uiaLysis)  (addeu) 

y.  None  of  die  above  (added) 

z.  Other: _ 

SOFTWARE  ENGINEERING  PROJECT  TE,\J-1 

59.  How  many  positions  [(filled  and  unfilled)]  supported  the 

project?  (added)  _ 

60.  Was  the  [software]  project  organization  divided  into  teams  each 
headed  by  a  technical  leader?  Note:  In  a  small  project  organiz¬ 
ation  with  diverse  activities  it  would  be  possible  to  have  one- 
person  teams.  (added) 


z.  Comment: _ 

61.  Whicli  types  of  teams  were  employed  in  software  development 
[(for  the  purpose  of  this  question  an  engineer  is  considered  an 
analyst) ]  (added) 

a.  Combined  [functional]  analysts-programmer  team  (added) 

b.  Separate  software  [functional]  analysis  team  (added) 


c.  Separate  programmer  team 

d.  Separate  test  team 

e.  Separate  integration  team 

f.  Separate  interface  team 

g.  Separate  product  acceptance  team  (added) 

y.  Did  not  employ  teams 

z.  Ocher: _ 

62.  Which  of  the  following  functions  were  internal  to  the  project 
management  organization? 

a.  Administration 

b.  Project  control 

c.  Program  support  librarian 

d.  Product/quality  assurance  (added) 

e.  Configuration  management  (added) 

y.  No  functions  of  this  type  were  integral  in  die  project 
management  organization 

z.  Others: _ 

63.  To  whom  did  the  test  team  report? 

a.  The  project  manager 

b.  Other  than  project  manager  (added) 

y.  Did  not  employ  a  [separate]  test  team  (added) 

z.  Other: _ 

64.  ow  many  individual  teams  were  assigned  to  the  project? 

65.  The  teams  were  organized  under: 

a.  A  chief  programmer  [Baker,  1972] 

b.  A  lead  programmer  (a  senior  e.xperienced  programmer) 

c.  Task  leader/work  ieader/project  leader  (added) 

d.  Lead  ergineer/systems  engineer /analyst  (added) 

y.  Did  not  use  teams 

z . 


Others : 
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66.  Were  any  of  the  positions  in  tiie  development  organ  i;’a  t  ion 
referred  to  by  the  following  titles?  [Titles  taken  from  Youruon, 
How  to  Manage  Structured  Programming.  1976,  and  Brooks,  My t!uca.l. 
Man-Month,  1975] 

a.  Chief  programmer 

b.  Back-up  programmer 

c.  Program  support  librarian 

d.  Surgeon 

e.  Administrator 

f.  Editor 

g.  Co-Pilot 

h.  Programming  clerk 

i.  Tool  smith 

j.  Tester 

k.  Language  lawyer 

l.  Software  architect 

m.  Software  engineer  (added) 

n.  Technical  director  (added) 

y.  None  of  the  above  (added) 


Other ; 


GENERAL 


67.  Which  of  the  many  forms  of  project  organization  do  you  feel 
contributes  the  most  to  the  success  of  the  project? 


68.  If  you  liad  it  within  your  power  to  make  one  ch.mg.e  in  the  w.iy 
the  project  was  organized,  wliat  action  would  you  take,  or  if  you  had 
tlie  resources  available  to  undertake  research  in  any  area  of  project 
organization,  which  aspect  would  you  explore? 


I 

I 


260 


SECTION  5  -  STAFFING 


PROJECT  MANAGER 

69.  What  was  the  source  of  the  project  manager? 

a.  New  hire  from  another  company 

b.  New  hire  from  school 

c.  Transfer  from  another  project 

d.  Transfer  from  within  company  otlier  Clian  project 

e.  Promoted  from  within  project  (added) 

f.  Tlie  project  manager  was  appointed  or  selected  for 

this  project  by _ 


y.  Did  not  have  a  project  manager  (added) 

z.  Otlier: _ 

70.  How  many  years  experience  did  tlie  project  manager  Have  in 
the  following  areas? 

a.  Functional  area  of  project 

b.  Data  processing 

•y.  No  experience  (no  project  manager)  (added) 
z.  Comment:  _ 
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71.  How  many  years  of  prior  experience  did  ciie  project  manager 
have  in  the  following  areas  and  capacities? 

A  3  C 


Capacity 


Area  Yrs 

a.  Commerciai/Business 

b.  Scientific 

c.  Simulation 

d.  Process  control  (to  include  _ 

embedded  computer  systems) 

e.  Command  and  control  _ 

f.  Mgc  information  system  _ 

g.  System  software  _ 

h.  Real  time  applications  _  _  _ 

i.  Data  communications  _ 

j .  Computer  operations  _ 

y.  No  experience/no  project  _ 

manager  (added) 

z.  Other: 


262 


72.  In  which  programming  languages  and  at  what  level  oi  p ro : ic  iency 


could  the 

project  manager  program? 

HIGH 

AVLIUGL 

:.ow 

a. 

FORTRAN 

[  ] 

[  ] 

[  i 

b. 

JOVIAL 

[  1 

;  ] 

[  ] 

c . 

COBOL 

[  I 

[  i 

t  i 

d. 

Assembler  [(unspecified)] 
(added) 

[  ] 

[  J 

i  1 

e. 

CMS-2  (added) 

[  ] 

[  1 

:  ] 

f . 

PL/1  (added) 

[  1 

[  ! 

!  1 

g- 

HOL  [(unspecified)]  (added) 

[  ] 

[  1 

<.  j 

y- 

None/no  project  manager 
(added) 

[  ] 

[  J 

[  ] 

z. 

Other ; 

[  ] 

[  ! 

i  I 

73.  The  age  of  project  manager  at  the  beginning  of  the  project 
was  _ years. 


74.  Was  the  project  manager  ever  a  Chief  Programmer  [As  defined 

by  Baker,  1972]? 

a.  Yes 

b.  No 

y.  No  project  manager  (added) 

z.  Comment: _ 

75.  Did  the  project  manager  receive  training  in  any  of  the  following 
areas  prior  to  (or  early  in)  the  project  development  cycle? 

a.  Functional  area  of  project 

b.  General  data  processing 

c.  Modern  programming  techniques 

d.  Project  management 

e.  General  management 

f.  Programming  languages  (give  language _ ) 

y.  None/no  project  manager  (added) 

z.  Other  project  related  area _ 

76.  Highest  education  level  attained  by  the  project  manager  was; 

a.  Less  than  high  school 

b.  High  school 

c.  AA  degree  or  two  years  of  college 


[  1 


[  1 


[  1 
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d.  Becween  2  and  4  years  of  college 

e.  BS/BA  degree 

f.  Masters  degree 

g.  Master  degree  plus  30  hours  or  doctoral  candidate 

h.  PHD  (or  equivalent) 

y.  Did  not  have  a  project  manager  (added) 

2.  Other: _ 

77.  If  the  project  manager  attended  college,  his  major  or 
speciality  was: 

a.  Computer  Science  [ 

b.  Mathmatics  [ 

c.  Engineering  [ 

d.  Physics  (added)  [ 

e.  General  Science  (added)  [ 

THE  SOFTWARE  DEVELOPMENT  STAFF 

78.  List  the  number  of  positions  authorized  by  category  over  the 
life  of  the  project,  and  list  next  to  chat  the  actual  number  of 
individuals  who  occupied  these  positions  (e.g.,  if  one  project 
manager  position  was  designated,  but  during  Che  course  of  the 
project  Chat  position  was  occupied  by  three  individuals  at  different 
times  Che  answer  would  be:  [1]  [3]). 


A  B 


Position 

Authorized 

Occupied  by 

a. 

Project  manager 

[  ] 

1  ] 

b. 

Asst  project  manager  (added) 

[  1 

[  1 

c. 

Functional  analyst/ (engineer ] 
(added) 

[  J 

[  1 

d. 

Data  processing  analyst 

[  1 

[  ] 

e. 

Programmer 

[  1 

[  1 

f . 

Support  librarian 

[  ] 

[  1 

g. 

Secretary 

[  1 

[  1 

h. 

Admin isCraCor 

[  1 

1  1 

i. 

User/Customer  (if  part  of 

Che  development  team) 

(  1 

[  1 

y- 

None  (added) 

[  I 

[  ] 

Z. 

Other : 

[  1 

[  ] 

L.  business 

g.  Liberal  Arcs 

y.  Did  not  attend/ 

no  project 
manager  (added) 

2.  Other: 
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79.  Whac  was  Che  source  (by  percent)  o£  Ciie  progranuner/anaiysc 
staff? 

a.  Mew  hire  from  another  company 

b.  New  hire  from  school 

c.  Transfer  from  another  project 

d.  Transfer  from  ocher  chan  another  project 

e.  ADP  staff  (added) 

f.  Subcontractor  (added) 

2.  Other: _ 

80.  At  wiiat  level  was  the  programmer  support  librarian? 

a.  Clerk/programmer  technician 

b.  Junior  programmer 

c.  Senior  programmer 

y.  Did  not  use  programming  support  librarian 
2.  Ocher: _ 

81.  The  chief  programmers  were: 

a.  Senior  programmers 

b.  Functional  area  experts 

c.  Specially  trained  for  task 

y.  Did  not  use  chief  programmers 

2.  Other: _ 

82.  Whac  was  the  education  level  of  the  programmer/analysc  (by 
percent)? 

a.  Less  than  high  school 

b.  High  school 

c.  t\A  degree  or  two  years  of  college 

d.  Between  2  and  4  years  of  college 

e.  BS/BA  degree 

f.  Masters  degree 

g.  Masters  degree  plus  30  hours  or  Doctorate  candidate 

h.  PHD  (or  equivalent) 

2.  Ocher: _ 

83.  What  percent  of  the  programmer/analysc  staff  were 
originally  computer  operators  who  moved  directly  from  machine 
operation  to  programming? 
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84.  Did  ciiese  former  operators  make  successful  programmers? 

(originally  narrative) 

a.  Yes,  excellent  [  ] 

b.  Yes,  good/average  [  J 

c.  Yes,  fair  [  ] 

d.  No,  poor  [  ] 

y.  Did  not  use  ex-operators  as  programmers  [  ] 

z.  Comment: _  [  J 

STAFF  SUPPORT 

35.  Were  the  following  services  provided  from  witiiin  the  project 
management  resources,  or  supplied  by  a  staff  function  within  ciic 
company?  (Indicate  by  inserting  "W"  for  within  tlie  project  resources, 

"0"  for  outside  the  project  resources,  ["B"  for  both])  (added) 


a. 

Personnel 

_  i . 

Train  ill)'. 

b. 

Accounting  _ 

S- 

Typing 

c. 

Budgeting  _ _ 

y- 

None  (added) 

d. 

e. 

TR^MNING 

Computer  Operation 

Travel  Arrangement 

_  2 . 

Other : 

86.  Was  the  project  manager  responsible  for  identifying  training 
requirement  of  the  development  team  members? 

a.  Yes 

b .  No 

y.  There  was  no  training  requirement  or  no  project 
manager  (added) 

z.  Other: _ 

87.  Training  requirements  generated  as  a  result  of  the  project 
were  satisfied  from  which  of  the  following  sources?  (Insert  0 
for  methods/sources  not  employed  and  rank  order  the  remaii.lng 
methods/sources  with  1  being  the  most  important.) 


a . 

On  tlie 

job  training 

b. 

Classes 

conducted 

by 

project  team  members 

c. 

Classes 

conducted 

by 

company  cadre 

d. 

Classes 

conducted 

by 

the  system  user 

e. 

Classes 

conducted 

by 

independent  training 

consultants 


1 


[  1 
(  I 
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f.  Classes  conducced  by  liardware/sot tware  vendors 

g.  Classes  conducted  by  colleges/universities 

y.  None  (added) 

z.  Ocher: _ 

88.  What  percent  of  the  personnel  assigned  to  die  project 
received  additional  training  in  the  programming  language 
selected? 

89.  What  percent  of  Che  personnel  assigned  to  the  project 
required  additional  training  on  the  operating  system  chat 
was  used? 

GENERAL 

90.  If  it  were  within  your  power  to  make  any  changes,  or 
initiate  any  research  in  the  area  of  staffing,  what  would  you 
consider  the  most  fruitful  area  for  modification  or  study? 
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SECTION  6  -  CONTROL 

PROJECT  CONTROL 

91.  Which  of  the  following  automated  or  manual  systems  were 


used  for 

project  control? 

a. 

PERT 

[  1 

g- 

Graphs/Rate  Charts 

(added) 

b. 

Modified  PERT 

[  ] 

h. 

Workloading 

c . 

[CPM]  (corr) 

[  I 

i . 

Milestone  tracking 

d. 

GANTT  Charts 

.  (  1 

y- 

No  project  control 

e. 

CSCS  (added) 

[  ] 

z. 

Other : 

f . 

WBS  (added) 

[  ] 

92.  Was 

a  work  breakdown 

structure 

code  used 

1  in  the  control  of 

system  development? 

a. 

Yes 

b. 

No 

z . 

Comment : 

REPORTING 

93.  Which  manual  reporting  procedures  were  used  in  project 
monitoring  and  management?  At  what  level  did  they  originate, 
and  how  high  did  they  go?  How  often  were  they  aggregated, 
condensed,  or  edited  as  they  moved  through  the  chain? 


A 

B 

C 

LOWEST 

HIGHEST 

NO.  OF 

REPORT  TITLE 

ORIGINATOR 

RECIPIENT 

AGGS/EUrS 

a.  Weekly/monthly 
Activity 

b.  Project  Status 

c.  Significant  Chg 

d.  Cost  vs  Performance 
(added) 

e.  _ 

f .  _ 

y.  None/Unknown  (added) 

z.  Other:  _ 
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94.  Which  auComaced  reporting  systems  were  used  in  project 
monitoring  and  management? 

6 

HlGHliST 
R£CI?IE.NT 

a.  Manhour  by  Activity  (e.g., 

code,  flow  diagram,  etc.)  _  _ 

b.  Manday  by  Task  (e.g., 
prepare  users  guide, 

design  data  base,  etc.)  _  _ 

c.  None/Unknown  _  _ 

2.  Other: 


A 

LOWEST 

ORICI.WATOR 


95.  In  monitoring  system  development,  system  software  was  used  to; 


a. 

Count  compiles  per  module 

[ 

b. 

Count  lines  of  code  produced 

[ 

c . 

Check  for  adherence  to  coding  conventions 

[ 

d. 

Check  for  use  of  standard  data 

element  names 

I 

e. 

Perform  manhour/cost  accounting 

(added) 

[ 

f. 

Check  performance  (added) 

[ 

g- 

Monitor  changes  (added) 

[ 

y- 

Did  not  use  software  to  monitor 

system  development 

[ 

Z. 

Other : 

[ 

96.  List  productivity  inde.xes  such  as  lines  of  code,  program 
errors,  sources  of  errors,  turn  arounds  required  per  completed 
task,  etc.,  that  were  employed  in  monitoring  performance. 
(Originally  narrative) 


a. 

Lines  of  code  (per  unit  of  time) 

[ 

b . 

Modules  compiled  (per  unit  of  time) 

[ 

c. 

Program  errors 

[ 

d. 

Computer  time  used 

[ 

e . 

Documented  pages 

[ 

y- 

z. 

Did  not  use  productivity  indexes 

Other : 

1 

[ 

2. 


269 


97.  Specific  standards  e.Tipioycd  during  the  course  of  tiie  project 
were  developed,  adapted,  or  directed  by: 


A 

s 

c 

Standard 

Project 

Manager 

Company 

L’se  r 

a. 

Data  names 

L  ] 

L  1 

[  ] 

b. 

Coding 

[  I 

[  I 

r  ^ 

i  J 

c . 

Programming 

[  ] 

[  ] 

r  ! 

1  i 

d. 

Documentation 

I  ] 

[  j 

L  1 

e. 

Testing 

[  1 

i  1 

i  i 

f . 

Management  report 

i  1 

[  ] 

[  ] 

g- 

Configuration  control  [  ] 
(added) 

[  ] 

[  ] 

h. 

Quality  assurance 
(added) 

(  ! 

[  ] 

[  ] 

y- 

No  standards  (added) 

[  1 

[  ] 

[  1 

Z. 

Other : 

1  1 

[  1 

[  1 

98.  Which  of  the  following  were  recognized  as  distinct  phases  in 
the  development  effort?  [AFR  300-15,  1978J 

a.  System  definition 

b.  Requirements  definition 

c.  Preliminary  design  (added) 

d.  Detail  design  (added) 

e.  System  design 

f.  Nodule  design 

g.  Coding 

h.  Module  test 

i.  Subsystem  .tegration 

j .  System  integration 

k.  System  test 

l.  Acceptance  (added) 

m.  Validation  (added) 

n.  Operation 

o.  Maintenance  (added) 

y.  Development  was  not  divided  into  [recognized]  piiases 

(added) 


D 

<iS 

None 


t 


j 


I  1 


I 

L  J 
[  ] 


z. 


other : 


f 
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AS S I G NMENT  OF  WORK 


99.  Task  assignmenc. 


a . 


b. 


c. 


d. 


e. 


r . 


y- 


z . 


Frequency 

AcCion 

Were  cask  assignmencs  Co  Che 
pro j ecc  c earns  given  in 
wricing? 

Were  required  complecion 
daces  included  wich  each 
wriccen  ceam  cask  assignmenc? 

Were  wriccen  ceam  cask 
assignmencs  prepared  in  such 
a  way  Chac  che  reiacionship 
of  Che  cask  co  Che  next  higher 
level  Cask  was  clearly 
delineaced? 

Were  cask  assignmencs  given 
CO  individual  ceam  members 
in  wricing? 

Were  required  complecion 
daces  included  wich  each 
individual  ceam  member's 
wriccen  cask  <»'‘signment? 

Were  individual  ceam 
members  wriccen  cask 
assignmencs  prepared  in  such 
a  way  chac  Che  reiacionship 
of  Che  cask  co  che  nexc 
higher  level  cask  was 
clearly  delineaced? 

No  Team  (added) 

Commenc : _ 


[  ]  [  ]  [  ]  [  ]  [1  [  1 

[  1  1  ]  [1  [  ]  I  1  [  ] 

[  1  [I  M  [  i  [1  M 

[  ]  [  ]  (  ]  [  ]  [1  [  ] 

[  ]  [1  [  ]  [  ]  [  ]  [  ] 

I  1 

_  [  1 
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100.  Were  cask  work  cime  estiniaces  provided  ac  cue  cinie  of  cask 
assignmenc  or  did  individual  ceam  members  provide  cime  escimaces 
afcer  reviewing  chis  assignmenc? 

a.  Time  and  Cask  assigned 

b.  Time  and  Cask  assigned  wich  verificacion  or 
modifi  acion  of  cime  aliocced  being  provided  by 
individual  Ceam  member  as  a  maccer  of  proeeaure 

c.  Individual  ceam  member  provided  che  cime  escimace  for 
each  Cask  assigned 

y.  No  efforC  was  made  co  decermine  cime  requirem.ancs  for 

individual  Casks 

z.  Ocher; _ _ 

101.  Ac  whac  incervais  were  Cask  assignmencs  made  co  individual 
ceam  members? 

a.  livery  5  work  days 

b.  Every  10  work  days 

c.  Every  monch 

d.  As  casks  were  developed  and  defined 

e.  As  resources  became  available  co  work  on  che  cask 

y.  Task  assignmenc  noc  made  co  individual  ceam  members 
(added) 

z.  Ocher: _ 

102.  Were  procedures  used  in  which  programmers  or  analyses  bid  on 

specific  casks  in  che  development  projects  (e.g.,  "I'll  write  the 
edit  program  for  $1,217.12")? 

a.  Yes 

b .  No 

z.  Comment: _ 

103.  If  bidding  as  described  above  was  employed,  how  successful 
was  it? 

a.  Very 

b.  Moderately 

c.  Unsuccessful 

y.  Not  employed 

z.  Useful  only  under  the  following  condiCion(s) 
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FORMAL  REVIE'.v’S 

104.  The  following  formal  reviews  were  conducced  as  a 
requirement  of  the  software  development  effort.  [AFR  100-15, 

1978] 

a.  Systems  requirements  review  f  j 

b.  Systems  design  review  j  ] 

c.  Preliminary  design  review  [  ] 

d.  Critical  design  review  ]  ] 

e.  Formal  qualifications  review  [  | 

y.  No  formal  reviews  were  conducted  [  ] 

z.  Other :  [  ; 

105.  CONCERNING  REVIEWS 


a.  Was  formal  documentation  provided  ]  j  [  j  1  i  [  j  (  j  [  : 

in  advance  of  each  review? 

b.  Did  reviews  take  place  on  [][][][][][] 

schedule? 

c.  Were  the  review  proceedings  [  ]  [  j  [  ]  [  j  [  ]  [  1 

formally  documented? 

d.  Did  top  management  attend  formal  [  ]  [  ]  [  ]  [  ]  [  ]  [  j 

reviews? 

e.  Did  tiie  customer/user  attend  [  ]  [  ]  [  ]  [  ]  [  ]  [  ] 

formal  reviews? 

f.  Was  an  independent  review  team  [  ]  [  J  [  j  [  j  (  ]  [  j 

used  (independent  of  project 

manager) ? 

g.  Did  the  project  manager  attend  [  ]  [  j  [  j  [  j  [  ]  [  ] 

the  formal  reviews? 

h.  What  was  the  title,  position  and  affiliation  of  tiie 

chairperson  of  the  formal  reviews? _ 


y- 


There  were  no  formal  reviews  (added) 
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CONFIGUI^MION  M\iN'AOEMENT 

106.  Was  a  software  configuration  management  system  empioyea? 

a.  Yes  (give  title/ name; _ 


z.  Comment; _ 

107.  The  system  was  base  lined  after;  [AFR  300-15,  1978] 

a.  System  requirements  review 

b.  System  design  review 

c.  Preliminary  design  review 

d.  Critical  design  review 

y.  Did  not  base  line  system  (added) 

z .  Other; _ 

108.  Were  formal  configuration  control  boards  employed? 


b.  No 


z.  Comment; _ 

109.  Give  title,  position  and  affiliation  of  chairperson  of 
Configuration  Control  Board.  (added) _ 


so [ tware . 


INFORMAL  REVIEWS  AND  WALK-THROUGHS 

110.  How  often  were  "informal"  reviews  conducted  between  tiie 
project  manager  and  his  supervisor? 

a.  Daily 

b.  Weekly 

c.  Monthly 

d.  As  required 

y.  There  were  no  informal  reviews 

z.  Other; _ 

111.  Walk-throughs  [as  defined  by  Weinberg,  1971]  wore  used  for; 

a.  Design  reviews 

b.  Coding  reviews 

y.  Did  not  use  walk-throughs 

z.  Other;  _ 


I 
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How  often  were 

wal.c-throughs 

scheduled? 

a.  Daily 

[  1 

d. 

As  required 

b.  Weekly 

[  ] 

y  • 

Not  used 

c.  Monthly 

.[  1 

Z  . 

Other : 

113.  Who  normally  accended  walk-tihroughs? 


a.  Pear  Programmers  or  analysts 

b.  Programmer  or  analyse  trainees 

c.  Programmer  or  analyst's  supervisor 

d.  Project  manager 

e.  Standards  monitor 

f.  Top  level  manager 

g.  User/customer 

h.  Quality  Assurance  (added) 

i.  Lead/Systera  engineer  (added) 

j.  Test  personnel  (added) 

y.  Did  not  use  walk-throughs 

z.  Other:  _ 


I  1 


114.  Were  walk-through  minutes/records  kept? 

A 

C/'  / 

Yes  [  ] 

No  [  ] 

Walk-throughs  not  used  [  ] 

Other:  [  1 


a. 

b. 

y- 

z. 


C 


•''C/ 

0/ 


(  1 


c 


t 


1  1 


[  1 
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GE.N'ERilL 

113.  If  it  were  within  your  power  to  make  any  changes  in  the  metlioa  or 
procedures  followed  in  controlling  this  project  what  would  these  changes 
be,  or,  if  you  had  resources  available  to  undertake  research  in  tiie  are.: 
of  planning,  which  aspect  would  you  explore? 
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SECTION  7  -  DIRECTING/MONITORING 

RESPONSIBILITY  AND  AUTHORITY  OF  THE  PROJECT  MANAGER 

116.  Enter  the  number  of  positions  (by  category  described) 
directly  involved  in  this  development  project. 


a. 

b. 

c . 

d. 

e. 

f . 

g- 

h. 


Full  time,  report  to  project  manager _ 

Full  time,  report  outside  project  manager's  orgn_ 

Full  time,  outside  contractor/consultant _ 

Full  time,  customer _ 


Part  time,  report  to  project  manager _ 

Part  time,  report  outside  project  manager's  orgn_ 

Part  time,  outside  contractor/consultant _ 

Part  time,  customer _ 


117.  The  project  manager  was  responsible  for: 
a.  Technical  quality 

Hire  and  fire  assigned  personnel  (within  firm  policy) 
Evaluate  performance  of  individual  personnel 
Administration,  budget,  etc. 

Allocating  computer  resources 
Allocating  non-computer  resources 
Meeting  schedule  commitments 

Negotiating  specification  changes  with  customer 
Making  a  profit  (i.e.,  operating  within  a  budget) 
Other : 


b. 

c. 

d. 

e. 

f . 

g- 

h. 

i. 
z . 


118.  The  title  and  position  of  the  project  manager's  supervisor  was: 


119.  The  span  of  control  of  the  project  manager's  supervisor 
(including  the  project  manager)  was _ persons. 


277 


120.  The  cicie,  posicion,  and  number  of  people  reporting  directly 
to  the  project  manager  was: 


A 

B 

C 

a. 

TITLE 

POSITION 

NUMBER 

b. 

c. 

d. 

e. 

f . 

g- 

Proiect  manager's  span  of  control  is  P 

ersons . 

y- 

None/Unkxiown  (added) 

[  1 

1  i 

121.  Indicate  the  reviews  the  project 
capacity  in  which  he  attended. 

manager  attended  and  the 

A  B 

c 

Reviewer  Reviewee 

Observer 

a. 

Formal  [management]  reviews 
(added) 

[  1  [  ] 

[  1 

b. 

Walk-throughs 

[  1  11 

[  1 

c . 

Budget  reviews  (added) 

[  1  [  i 

[  1 

d. 

Technical  reviews  (added) 

[1  11 

[  1 

y< 

Did  not  attend/ there  were 
no  reviews  (added) 

[  1  [  1 

[  ] 

Z  . 

Ocher : 

[  1  [1 

[  ] 

122.  The 

CO  be : 

project  manager  was  expected 
(added) 

[by  corporate  management] 

a. 

A  technical  supervisor 

[  1 

b. 

A  non-cechnical  supervisor 

[  ] 

c . 

Project  administrator  [only] 

(added) 

[  ] 

y- 

None  of  Che  above  (added) 

[  1 

Z  . 

Comment : 

[  . 
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MANAGEMENT  AND  MANAGEMENT  TECHNIQUES 


123.  In  the  conduct  of  this  project  did  the  company  or  project 
manager  make  a  conscious  effort  to  apply  any  of  the  following 
management  techniques? 


a. 

Management  by  objectives  [Drucker, 

1954] 

[ 

b. 

Job  enrichment  [Herzberg,  1977] 

[ 

c. 

[Motivation]  theories  [Maslow,  1943] 
(corr ) 

[ 

d. 

Suggestion  program 

[ 

e. 

Incentive  and/or  bonus  program 

[ 

f. 

Participative  management 

[ 

g- 

Management  by  exception 

[ 

y* 

None  of  the  above  (added) 

[ 

Z  . 

Others : 

[ 

A 

Company 
[  ] 

[  ] 


Pro  !  Mgr 


1  i 


[  1 


124.  In  using  management  by  objectives  the  objectives  were; 


a. 

Set  by  the  project  manager  without  input 
team  member 

from  tlie 

[ 

b. 

Set  by  the  project  manager  with  input  from  the 
team  member 

[ 

c . 

Periodically  reviewed 

[ 

y- 

Did  not  use  management  by  objectives 

[ 

z. 

Comment : 

[ 

The 

project  manager: 

a. 

Issued  instructions  to  his  subordinates  : 

in  writing 

[ 

b. 

Delegated  technical  decisions  to  his  direct  subordinates 
(Team  chiefs) 

[ 

c. 

Had  an  "open  door"  policy 

L 

d. 

Employed  quality  standards  defining  what 
of  each  programmer/analyst 

was  expected 

[ 

e. 

Employed  quantity  standards  defining  how 
programmer/analyst  should  accomplish 

much  each 

[ 

f. 

Monitored  progress  with  the  aid  of  a  control  board 
and/or  control  room 

[ 

g- 

Had  a  separate  office  to  insure  privacy 

[ 

h. 

Had  a  personal  secretary 

( 

[  ] 


[  1 


[  ] 


(  ] 


126.  How  did  che  union  contracC  affecc  Che  responsibiiicy  and 
auchoricy  of  che  projecc  manager? 

a.  Limiced  his  abilicy  co  hire  and  fire  his  scnif 

b.  Required  clearance  from  die  union  prior  co  caiing 
cercain  managemenc  accions 

c.  Union  helped  moCivace  ceam  personnel  coward  beccer 
performance 

d.  Union  labor  sCandards  recarded  Che  projecc  managers 
auchoricy 

f.  Union  labor  scandards  enhanced  produce  ion 

g.  Union  labor  scandards  recarded  production 

h.  Union  membership  had  no  recognizable  affecc  on  che 
project 

y.  No  ceam  members  were  represented  by  a  union 

2.  Other _ 

GENERAL 


127.  If  you  had  ic  wichin  your  power  to  implement  changes  in  che  way 
this  projecc  was  directed,  or  had  che  resources  Co  devote  co  research 
in  this  area,  whaC  action  would  you  cake? 


SECTION  8  -  DELIVERABLES  AND  SUCCESSES 

INTRODUCTION.  This  seccion  of  che  survey  presupposes  chat  the 
project  being  reported  on  is  complete  or  very  nearly  complete. 

The  following  question  tests  this  supposition  and  provides 
qualifying  data  for  the  remaining  inquiries. 

128.  This  project  is: 

a.  50%  complete  or  less 

b.  51%  to  75%  complete 

c.  76%  to  90%  complete 

d.  91%  to  98%  complete 

e.  99%  to  99.99%  complete 

f.  100%  complete  and  accepted  by  customer 

y.  Cancelled/not  deliverable  (added) 

z.  Comment: _ 

Rather  than  provide  two  sets  of  questions  we  ask  you  to  restructure 
the  question  in  your  mind  to  relate  them  to  present  project  status. 
In  the  question:  "Was  the  project  completed  on  or  before  schedule? 
could  be  read  "will  the  project  be  completed  on  or  before  schedule? 
and  the  degree  of  forecast  derived  from  your  just  stated  position 
on  the  development  continuum. 

MEETS  SCHEDULE 

129.  Was  the  project  completed  on  or  before  schedule? 


a. 

Yes  on  schedule 

b. 

Yes,  months  early. 

%  early 

c. 

No,  months  late. 

%  late 

d. 

Yes,  however  specifications 

were  reduced 

(added) 

y- 

Project  not  completed  and/or 
(added) 

no  estimate 

available 

z. 

Other : 

130.  If  not  completed  on  rime  what  was  the  major  cause  of  slippage 
[/cancellation]?  (added) 

a.  Cliangc  in  requirement 

b.  Change  in  method  of  design 

c.  Bad  initial  estimate 

d.  Unreasonable  estimate,  by  either  top  management  or  tlie 
customer 

e.  Limited  authority  over  resources 
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f.  Excessive  absences  on  Che  pare  of  the  project  team 
members 

g.  Poor  management  (added) 

y.  Project  was  completed  or  is  on  schedule  (added) 

z.  Ocher: _ _ _ 

132.  If  project  was  late,  what  portion  (in  percent  is 
attributable  to  change  in  requirements? 

133.  What,  if  anything,  could  Che  project  manager  have  done  to 
improve  his  ability  to  meet  the  schedule?  (Originally  narrative) 

a.  Restrict  the  number  of  changes 

b.  ■  Require  formal  system  of  clianges 

c.  Require  a  firm  requirements  baseline 

d.  Conduct  more  reviews 

e.  Receive  more  authority  over  project 

f.  Made  a  better  estimate  of  schedule 

g.  Perform  better  planning 

h.  Nothing 

i.  Better  direction  and  control 

j.  Better  and  earlier  requirement  specifications 

y.  Project  was  completed  or  is  on  schedule 

z.  Ocher: _ 

MEETS  COST 

13d.  Was  the  project  delivered  within  the  original  budget? 

a.  Yes,  on  cost 

b.  Yes,  $ _ funder  cost,  _ Z  under  cost 

c.  No,  $ _ ^over  cost,  _ Z  over  cost 

d.  Yes,  however  specifications  were  reduced  (added) 

y.  Project  not  completed  and/or  no  estimate  available 
(added) 

z.  Ocher: _ _ 

135.  If  not  completed  within  the  original  budget,  wliat  was  Che 
cause  of  this  slippage  [/cancellation]?  (added) 

a.  Change  in  requirements 

b.  Change  in  method  of  design 

c.  Bad  initial  estimate 
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d.  Unreasonable  escimace,  by  either  the  manager  or  the 
customer 

e.  Limited  authority  over  resources 

f.  Excessive  absences  on  the  part  of  the  project  team 
members 

Poor  management  (added) 

y.  Project  was  completed  or  is  within  cost  (added) 

z.  Other: _ 

136.  If  project  exceeded  cost  estimates,  what  portion  of  the  overrun 

(in  percent)  was  caused  by  a  change  in  requirements?  _ 

137.  What,  if  anything,  could  the  program  manager  have  done  to 
improve  his  ability  to  meet  the  budget?  (Originally  narrative) 

a.  Restrict  the  number  of  changes 

b.  Require  formal  system  of  changes 

c.  Require  a  firm  requirements  baseline 

d.  Conduct  more  reviews 

e.  Receive  more  authority  over  project 

f.  Made  a  better  estimate  of  costs 

g.  Perform  better  planning 

h.  Nothing 

i.  Better  direction  and  control 

j.  Better  and  earlier  requirement  specifications 

y.  Project  was  completed  or  is  within  cost 

z.  Other:  _  _ _ _ 


[  J 


[  ] 


[  ] 


MEETS  REQUIREMENTS 

138.  Did  the  delivered  software  meet  the  requirements  as  originally 
specified  by  the  customer? 

a.  Yes  [  ] 

b.  No  [  ] 

y.  No  requirements  specified  [and/or  project  not  completed]  [  j 

(added) 

z.  Other: _  [  1 

139.  Did  the  customer  accept  the  delivered  system  as  meeting 
specified  requirements  in  total,  or  did  he  identify  exceptions? 

a.  Accepted  system  in  total  as  meeting  specified  [  ] 

requirements 
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b.  Identified  exceptions 

y.  No  requirements  specified  [and/or  project  not  completed] 

z.  Other: _ 

140.  How  was  it  determined  that  the  system  met  the  requirements 
specified?  (Originally  narrative) 

a.  Qualif ication/field  test 

b.  System  in  operation 

c.  Formal  system/acceptance  test 

d.  Independent  verification  and  validation 

e.  Check  against  known  answers 

f.  Simulation 

g.  System  did  not  meet  requirement  specified 

y.  No  requirements  specified  and/or  project  not  completed 

z.  Comment: _ 

141.  Why  did  the  system  not  meet  the  required  specification? 
(Originally  narrative) 

a.  Requirements  changed 

b.  Initial  estimate  bad 

c.  Requirements  ignored/customer  accepted  without 
correction  (added) 

d.  System  met  requirements  specified 

y.  No  requirements  specified  and/or  project  not  completed 

z.  Comment: _ 

^iEETS  RELIABILITY  STANDARDS 

142.  Did  the  delivered  software  meet  the  reliability  standards 
originally  specified? 

a.  Yes 

b.  No 

y.  No  standards  specified  [and/or  project  not  completed] 
(added) 

z . 


[  ] 
[  1 
L  i 


L  i 
[  1 


[  1 


Comment : 
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143.  Why  did  the  delivered  software  fail  to  meet  the  reliability 
standards? 

a.  Requirements  changed  (added)  [ 

b.  Not  enough  reviews  and  testing  (added)  [ 

c.  _  t 

d.  System  met  the  specified  reliability  standards  i 

y.  No  standards  specified  [and/or  project  not  completed]  [ 

(added) 

z.  Comment; _  [ 

144.  Which,  if  any,  of  the  following  methods  of  measuring 
reliability  of  the  delivered  software  system  were  used  in  tne 
project?  [Gilb,  1977] 

a.  The  finished  system  was  "salted"  with  k,nown  bugs  (after  [ 

integration  testing  and  before  sign  off).  The  system 

was  then  debugged  for  a  fixed  (given)  period  of  time 
after  which  the  ratio  of  found  known  bugs  to  found 
unknown  bugs  was  judged  equal  to  the  ratio  of  unfound 
known  bugs  to  unfound  unknown  bugs. 

b.  Curves  of  errors  found  to  time  spent  debugging  were  [ 

calculated. 

c.  A  software  reliability  error  prediction  model  was  [ 

employed. 

d.  Special  test  drivers  were  employed  to  stress  system  [ 

(added) . 

e.  Reported  errors  per  uri:  of  time  (MTBF)  and  system  [ 

accepted  when  error  r  .  reached  acceptable  level 

(added) . 

y.  No  reliability  measuring  method  was  employed  [and/or  [ 

project  not  completed]  (added). 

z.  Other: _ [ 


MEETS  RELIABILITY  STrVNDARDS 

145.  Did  the  delivered  software  meet  the  maintainability  standards 


originally  specified? 

a.  Yes  [ 

b.  No  [ 

y.  No  standards  specified  and/or  project  not  completed  [ 

(added) 

z.  Comment: _ [ 
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i-i6.  Why  did  Che  delivered  software  fail  to  meet  the  maintainability 
standards  ? 


a.  _ 

b .  _ _ 

c .  _ 

d.  Software  met  the  specified  maintainability  standards 

y.  No  standards  specified  [and/or  project  not  completed] 
(added) 

z.  Comment: _ 

117.  Which,  if  any,  of  the  following  methods  of  measuring  t:ie 
maintainability  of  the  delivered  software  were  usea  in  tiie  orojecc? 
[Gilb,  1977] 

a.  Measured  die  time  Co  repair  the  first  "N"  number  of  bugs 
reported  (after  integration  testing  and  just  before  sign 
off).  The  average  time  to  find  a  bug  became  tiie  mean 
time  to  repair  (MTTR) . 

b.  "Salted"  the  finished  code  with  known  bugs.  i'he  average 
time  CO  find  and  repair  Che  known  bugs  became  the  mean 
time  to  repair  (MTTR) . 

c.  Cost  to  maintain  program  per  unit  of  time  and  accepted 
when  cost  reached  ar  acceptable  level  (added) 

y.  No  maintainability  measures  were  employed  and/or  project 
not  completed  (added) 

2,  Other: 


MEETS  USABILITY  REQUIREMENTS 

118.  Was  Che  delivered  system  usable?  If  so,  how  was  this 


decerma 

nation 

made?  (Originally  narrative) 

a. 

Yes , 

system  is  in  production 

b. 

Yes , 

system  is  in  use 

c. 

Yes , 

independent  verification  and  validation 

d. 

Yes, 

field  or  qualification  testing 

e. 

Yes , 

customer  was  satisfied 

f . 

Unknown,  has  not  been  determined 

g- 

No, 

user  was  not  satisfied 

y , 

No, 

Che  system  was  not  delivered/cancelled 

z. 

Ocher : 

I 
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GENERA 

149.  What  was  the  period  of  the  final  product  warranty? 


a. 

1  year 

[  ]  d. 

6  or  more  years 

[ 

b. 

2  years 

[  ]  y. 

No  warranty 

[ 

c. 

3-5  years 

[  ]  z. 

Other : 

[ 

150.  What  was  the  approximate  production  rate  (lines  of  code  per 
day)  for  the  entire  project? 

a.  Average  programmer/analyst 

b.  Best  programmer/analyst 

c.  Worst  programmer/analyst 

z.  Comment  _ [  ] 

151.  How  many  lines  of  code  were  produced  per  page  of 
documentation? 

152.  What  was  the  cost  per  line  of  code?  $ 

153.  Was  software  development  the  pacing  factor  (critical  path) 
on  the  project? 

a.  Yes,  hardware  was  developed  ahead  of  software  [  ] 

b.  No,  software  was  developed  ahead  of  hardware  [  J 

c.  Yes,  project  was  almost  all  software  [  ] 

d.  No,  hardware  and  software  delivered  together  (added)  [  ] 

e.  Varied,  hardware  and  software  alternated  (added)  [  ] 

y.  Project  not  completed/there  was  no  pacing  factor  (added)  [  J 

z.  Other _ _  [  ] 

154.  What  percent  of  production  [calendar  time]  was  spent  in  the 


following  areas?  (added) 

a.  Requirement  specification  ^ 

b.  Preliminary  design  Z 

c.  Detail  design  Z 

d.  Programming/ unit  testing  Z 

e.  Integration  testing  Z 

f.  System  testing  ^ 

y.  Project  not  completed  (added)  [  ] 

z.  Comment :  _ _ _ [  ] 
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155.  If  Che  projecc  used  on-line,  inceractive  programming  for 
program  development,  was  it: 


a. 

A  highly  effective  development  tool 

[ 

b. 

Effective  in  some  cases 

[ 

c . 

Of  limited  utility 

[ 

d. 

A  drain  on  hardware  resources 

r 

1 

e. 

A  nice  toy 

[ 

y. 

On-line,  interactive  program  was  not  used/unknown 

[ 

z . 

Other : 

[ 

156.  If 

your  experience  reflects  that  on-line,  interactive 

programming  was  an  effective  tool,  in  wh-ich  situation  was  it 
most  effective?  (Check,  or  rank  order) 


a. 

During  development  of  code 

[ 

b. 

To  Cry  short  length  of  code  for 
(simulation  approach) 

possible  use 

[ 

c. 

During  debugging 

[ 

d. 

During  testing 

[ 

e. 

Not  an  effective  tool 

f 

L 

y- 

Did  not  use  on-line,  interactive 

programming/ unknown 

[ 

z. 

Ocher : 

[ 

157.  If  your  experience  reflects  that  on-line,  interactive 
programming  was  an  effective  tool  what  do  you  feel  the  improvement 
in  programmer  productivity  over  conventional  (batch)  software 
development  was? 


a. 

Not 

an  improvement 

b. 

1.5; 

:1  improvement 

c. 

2:1 

improvement 

d. 

3:1 

improvement 

e. 

5:1 

improvement 

y- 

Did 

not  use  on-line 

z . 

Ocher : 

[  J 
[  ] 
[  J 
[  ] 
[  ] 
[  1 


158.  Overall,  how  well  do  you  think  chat  this  project  met  the  project 
managers  major  goals:  to  deliver  on  time,  within  budget,  and  meeting 
Che  requirement  of  the  system,  where  the  final  software  product  is 
reliable,  maintainable,  and  usable? 


a. 

Extremely  well 

[  ] 

e. 

Poor 

b. 

Very  well 

[  ] 

f . 

Failed 

c. 

Good 

(  ] 

z. 

Ocher : 

d. 

Fair 

[  ] 

159.  Give  some  lessons  learned  from  this  project. 
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INTRODUCTION .  This  part  of  the  survey  contains  questions  that  are 
general  in  nature  and  pertain  to  software  development  projects  as  a 
and  were  designed  to  be  answered  by  the  project  manager. 

(These  questions  were  originally  1,  2,  3,  4,  and  25  of  Section  Thre 

160.  Place  the  following  objectives  of  a  project  manager  in  order  o 
importance.  (Enter  number  1  through  6  in  the  space  provided)  The 
project  should; 


a. 

Be  within  budget 

[  ] 

f . 

Be  usable 

b. 

Be  on  schedule 

[  ] 

g- 

Depends  on 
contract  incentiv 

c . 

Meets  requirements 

t  ] 

(added) 

d. 

e. 

Be  maintainable 

Be  reliable 

[  ] 

[  1 

h. 

Varies  with  job 
(added) 

161.  It 

is  reported  in  literature 

that : 

1  support 

.  librarian,  plus 

programmers  can  do  the  work  of  3 

programmers.  Do 

you  believe  this? 

a.  Yes 

b.  No 

c.  Do  not  know,  insufficient  data/experience  (added) 

2.  Other:  _ 


162.  If  you  believe  the  support  librarian  does  relieve  the 
programmer/analysc  of  unnecessary  tasks,  but  do  not  believe  the 


ration  is 

2  to  1,  what 

ratio  do  you  consider  mo 

re  accurate? 

a. 

1  to  1 

[  ]  £. 

Varies,  according 
to  skill  level 

b. 

3  to  1 

[  1 

(added) 

c . 

o 

U 

[  1  g. 

Do  not  believe/do 
not  know  (added) 

d. 

5  to  1 

[  ] 

y- 

Believe  2  to  1 

e. 

10  to  1 

(  ] 

z. 

Other : 

163.  What 

actions  are 

customarily  taken  by  the 

project  manager  when 

it  is  discovered  that 

a.  Wring  hands 

a  project  is  behind  schedule? 

b. 

Assign  more 

personnel  to  the  project 

c . 

Renegotiate 

schedule  with  customer 

d. 

e. 

Renegotiate 

Quit 

schedule  with  manager 

whole , 
0 

[  ] 

[  1 


[  1 
[  ] 
I  1 
[  i 

[  ] 

[  ] 

[  1 
[  ] 

[  ] 
[  ] 
[  ] 
[  1 
[  1 
[  ] 


f.  Say  nothing  to  top  management  hoping  to  make  up 
schedule  time 
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g.  Reduce  projeccs  goals  (added) 

h.  Realign  manpower  (added) 

i.  Use  overtime  (added) 

j.  Replan  the  project  (added) 

k.  Varies,  depends  on  situation  (added) 

y.  Nothing  (added) 

z.  Other; _ _ 

164.  Please  furnish  any  additional  comments  or  statement  concerning 
this  survey  or  the  science  of  software  engineering  project 
management . 
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APPENDIX  C 

COMMENTS  ON  AND  ABBREVIATIONS  USED  IN 
THE  REDUCTION  OF  ANSWERS 

INTRODUCTION 

The  purpose  of  this  Appendix  is  to  present  comments  on  specific 
questions,  relationships  between  questions  and  their  answers,  procedures 
used  in  contriving  missing  answers,  and  to  list  and  describe  the 
abbreviations  and  codes  used  in  this  report.  In  addition,  since  a  few  of 
the  questions  were  poorly  written  and/or  in  general  misunderstood,  some 
comments  along  these  lines  will  also  be  reported  here. 

To  conserve  space  and  to  provide  a  means  of  using  a  computer  for 
analysis,  all  answers  were  abbreviated  and/or  coded  (abbreviations  and 
codes  will  be  called  just  codes  for  the  balance  of  this  report).  Because 
of  space  limitations  and  to  assist  in  ease  of  processing,  all  alpha¬ 
numeric  codes  were  restricted  to  exactly  three  characters.  The  use  of 
codes  also  had  an  additional  advantage;  it  effectively  disguised  the 
answers  so  tliat  the  participants  continue  to  remain  anonymous. 

The  authors  did  not  comment  on  ail  the  questions  and  answers.  If  the 
authors  have  a  comment,  discussion,  or  observation  on  a  question,  their 
comments  immediately  follow  the  question  number.  Codes  will  immediately 
follow  comments.  If  there  are  no  comments,  the  cooes  will  follow  the 
question  number.  If  the  authors  have  no  comment  or  codes  concerning  a 
given  question,  the  question  number  will  be  passed  by. 

Four  types  of  codes  were  used.  The  first  type  was  general,  applies  to 
all  questions,  and  will  be  defined  after  this  introductory  section.  The 
second  type  was  applicable  to  only  one  answer  and  appears  after  the 
question  number  in  this  section.  The  third  type  of  code  was  general 
across  two  or  more  questions  (e.g.,  FOR  for  FORTRAN;  GPC  for  general 
purpose  computer),  and  was  listed  once  the  first  time  it  is  used.  There¬ 
fore,  all  Type  Two  and  Three  codes  were  defined  after  the  question  number 
in  which  they  first  appear.  The  code  was  separated  from  its  explanation 
by  a  dash  ( — ) . 

The  fourth  code  type  applied  to  questions  33,  51,  52,  67,  68,  90,  115, 
127,  159  and  164  only.  These  questions  were  strictly  narrative  in  nature 
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and  did  not  lend  themselves  to  multiple  choice.  To  code  these  questions, 
all  narrative  sentences  were  broken  into  phrases  generally  containing  a 
verb  and  object.  These  phrases  were  then  given  a  code.  The  codes  indi¬ 
cate  whether  the  phrase  concerned  a  planning,  organizational,  staffing, 
control,  etc.,  function. 

The  authors  attempted  to  use  codes  that  were  easy  to  recognize 
(mnemonic)  to  reduce  the  amount  of  flipping  between  appendixes. 

The  letters  a  through  z  indicated  the  sub-parts  of  the  questionnaire. 
Parts  a  through  w  were  general  questions.  Part  x  was  used  to  indicate 
the  entire  question  was  not  answered  (ie. ,  skipped) .  Part  y  was  used 
primarily  to  show  "none"  of  the  answers  applied  or  the  questions  were 
"not  applicable".  Part  z  was  used  for  "other"  answers. 
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TYPE  ONE  CODES 

The  code  "YES"  on  Che  lisCing  opposite  a  question  (Sub-parts  a 
through  w)  indicated  that  the  surveyee  "checked"  the  answer  without 
comment  and  the  answer  is  "yes"  or  "true".  If  a  given  question  has  a 
"blank."  for  an  answer  this  indicates  that  Che  surveyee  answered  "no",  or 
that  Che  answer  is  "false"  (this  cannot  be  assumed  if  the  surveyee  did 
not  answer  at  least  one  part  in  a  multiple-part  question)  . 

Sometimes  a  pseudo  question.  Sub-part  x,  was  created  to  indicate  that 
the  surveyee  did  not  provide  an  answer  to  a  given  question  because  he: 

1)  did  not  understand  the  question,  2)  felt  it  did  not  apply  to  his 
project  or  organization,  or  3)  just  did  not  feel  like  answering  it. 

This  was  done  so  that  the  reader  would  not  read  a  "no"  when  the  correct 
answer  was  unknown  to  the  authors.  Sometimes  the  surveyee  wrote  in 
"unknown"  otherwise  it  was  coded  "MIS"  by  the  authors. 

When  answered,  Sub-part  y  was  coded  "NON"  to  mean  chat  the  whole 
question  was  answered  "no"  or  "none".  The  answer  to  Sub-part  y  was 
frequently  supplied  by  the  authors,  therefore,  one  of  die  "C"  codes  was 
used  (see  later  discussion) .  Sub-part  z  was  coded  "OTH"  to  mean  that  the 
surveyee  wrote  in  another  answer  and  the  authors  were  not  able  to  use  it 
any  ocher  way  (see  discussion  Appendix  B). 

As  an  added  note,  answers  to  Sub-parts  a  through  w  and  z,  Sub-part  .x 
and  Sub-part  y  are  mutually  exclusive. 

Several  of  the  questions  are  multi-part.  It  is  assumed  Chat  if  a 
surveyee  answered  any  one  part  of  the  multi-part  question  "yes"  or  "true" 
then  all  parts  of  Che  questions  were  answered.  Any  answers  that  were 
not  checked  were  "no"  or  "false". 

Other  Type  One  codes  are  listed  below.  These  codes  were  frequently 
used  when  the  surveyee  did  not  answer  a  question  but  made  some  comment 
in  Che  margin.  Ocher  times  Cliese  codes  were  used  as  the  appropriate 
answer  to  a  narrative  question. 

CFU  —  Confusing,  vague,  don't  understand 

DEL  —  Deleted  by  authors  as  revealing  Che  participant 

DIS  —  Disagree  with  question  as  written,  not  true  statement 

D/K  —  Don' C  know 
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INF  —  Infinite,  continuous 

N/A  —  Not  applicable  (on  this  project),  didn't  use 
NEG  —  Negative,  non-contributary 
NON  —  No,  none,  or  false 
NOS  —  No  solution  (yet) ,  hard  to  solve 
N/S  —  Not  specified/not  selected 
-OTH  —  Other 

UI'iK  —  Unknown  (also  included  "?"  as  an  answer) 

VAR  —  Variable 

MIS  —  Question  not  answered  (supplied  by  authors) 

YES  —  Yes  or  true 

Upon  occasion  the  authors  felt  it  necessary  to  either  answer  the 
question  for  the  surveyee,  or  change  his  answer.  In  the  interest  of 
honest  reporting,  the  following  codes  indicate  wtietlier  or  not  the  answer 
was  changed/contrived  and  the  reason.  These  change  codes  were  COi ,  C02, 
C03,  and  C04.  COI  has  the  highest  probability  that  the  changed  answers 
reflect  the  true  answer,  C02  next  highest  probability,  C03  next,  and  C04 
the  lowest  probability. 

The  change  codes  follow: 

COI  —  This  answer  was  supplied  by  the  authors  and  the  answer 
chosen  was  based  on  an  answer  to  a  different  question,  e.g.,  if  the 
surveyee  answered  Question  3  with  any  answer  but  c,  and  he  did  not 
answer  Question  4,  answer  4y  was  provided  by  the  authors  as  COi. 
Again,  if  the  surveyee  answered  Question  46A  but  not  46B,  answer 
46By  was  coded  COI  by  the  authors. 

C02  —  These  answers  were  manufactured  by  the  authors  by  compart¬ 
mentalizing  answers  provided  originally  in  narrative  form,  i.e., 
multiple  choice  answers  were  formulated  after  all  the  answers  were 
supplied  by  the  participants.  These  answers  were  originally  in 
narrative  type  questions  or  answers  provided  under  "comments"  or 
"other,"  j\n  exception  to  this' was  when  the  participant  wrote  in 
"none"  or  another  negative  comment  because  a  "none  of  the  above" 
type  answer  was  not  supplied  by  tiie  autliors.  When  this  happens 
a  "none"  answer  is  manufactured  but  coded  COI. 

C03  —  These  answers  redirected  by  the  authors  from  the  one  given 
by  the  surveyees  as  "other"  or  "comment"  to  one  of  the  existing 
answers  which  the  authors  felt  was  equally  as  good  as  the  one  placed 
in  "comment",  e.g.,  in  Question  2  one  of  the  answers  placed  under 
"other"  was  "airborne  computer."  The  authors  deleted  the  "other" 
answer  and  substituted  code  C03  in  answer  2e.  This  was  done  to 
reduce  the  number  of  possible  answers  while  still  retaining  as  much 


accuracy  as  possible.  C03  was  also  used  Co  answer  a  question  chat 
was  redundant,  i.e.,  the  question  appeared  T.orc  tiian  one  ciir-.e  and 
was  only  answered  once.  The  unanswered  question  was  answered  by 
Che  authors  as  C03,  e.g.,  if  Question  34e  was  checked  but  55a  was 
not,  answer  55a  was  coded  C03  by  the  authors. 

C04  —  These  answers  were  redirected  by  the  auciiors  from  a  supplied 
answer  to  anociier  answer.  This  was  only  cone  witii  ^tmo.-,t  c.iution. 
It  was  done  only  if  the  supplied  answer  was  not  reasonaoie  and 
there  was  reason  to  believe  it  was  incorrect. 


TYPE  TWO AND  TYPE  THREE  CODES 


These  Sections  reflect  the  specific  codes  and  comments  concerned  with 
each  question  and  are  ordered  by  the  question  number.  If  there  is  no 
code  or  comment  necessary  for  a  given  question,  it  is  skipped. 

SECTION  1  -  project  IDENTIFICATION 

3-4.  Question  4  is  dependent  on  Question  3.  If  the  participant  answered 
Question  3a,  b,  or  z  and  did  not  answer  Question  4,  answer  4y  was  coded 
COi. 

5-12.  Questions  6  through  9  are  dependent  on  the  answer  to  Question  5. 

If  Question  5  was  answered  5c  through  5z  (any  or  all)  aiid  Questions  6, 

7,  3  or  9  were  not  answered,  answer  y  was  coded  COI.  Questions  10 
through  12  are  also  dependent  on  the  answers  to  Question  5.  If  Question 
5  was  answered  5a  and/or  5b  and  Questions  10,  11  or  12  were  not  answered, 
answer  y  was  coded  COI. 

8.  Answers  to  Question  8a  were  divided  into  al,  a2,  a3,  and  a4 .  Answers 
to  Question  8b  were  divided  into  bl,  b2,  b3,  and  b4 .  Answers  a3,  a4 ,  b3 
and  b4  were  only  used  if  two  machines  and/or  operating  systems  were 
reported . 

The  following  codes  were  used  for  answers  8ai,  8a3,  8bl,  and  8b3 
(manufacturer)  if  the  manufacturer  was  given: 

CDC  —  Control  Data  Corporation 

DEC  —  Digital  Equipment  Corporation 

GEC  —  General  Electric  Corporation 

HCS  —  Harris  Computer  Systems 

HIS  —  Honeywell  Information  Systems 

HPK  —  Hewlett-Packard  Corporation 

IPM  —  International  Business  Machines 

INH  —  In-house 

LIT  —  Litton  Corporation 

SDC  —  Systems  Development  Corporation 

SEL  —  System  Engineering  Laboratory 

UNI  —  Sperry-Univac  Corporation 

ZER  —  .XEROX  Corporation 

The  following  code  was  used  for  Answers  8a2  and  8a4  (computer  make 
and  model)  if  the  computer  type  was  given: 

GP#  —  Number  which  follows  "GP"  was  taken  from  tiie  size  class 
numbers  reported  under  "General  Purpose  Computer  Census",  Computer-World 
(Jun  1978) ,  a  copy  which  follows  this  appendix. 
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The  following  codes  were  used  for  Answers  8a2  and  3a4  (computer  make 
and  model)  if  the  computer  type  was  not  given  ■'r  the  code  did  not  appear 
in  Che  "General  Purpose  Computer  Census": 

GMI  —  General  purpose,  mini-computer 

SLG  —  Special  purpose,  large  computer 

SMI  —  Special  purpose,  mini-computer 

The  following  codes  were  used  for  Answers  8b2  and  884  (type  operating 
systems)  if  the  type  operating  system  was  given.  Many  of  tlie  answers 
were  abbreviated  and  not  always  completely  understood  by  the  autiiors. 
Answers  are  given  as  reported: 

DVS  —  DOSVS 

GCO  —  GCOS  (GECOS) 

IMS  —  IMS 

OSM  —  OSNfNT 

OVT  —  OSMVT 

RPM  —  RPM 

RSX  —  RSX 

RTE  —  RTE 

RTM  —  RTM 

SCO  —  SCOPE 

SVS  —  SYS  IMS 


SYM  —  SYMON 
SYS  —  SYS  II 

The  following  codes  were  used  for  Answer  8b2  and  3b4  (type  operating 
systems)  if  Che  name  of  the  operating  system  was  not  given: 

GOS  —  General  purpose  operating  system 

SOS  —  Special  purpose  (one  of  a  kind)  operating  system 

9.  The  codes  used  for  Che  answers  to  this  question  were  the  same  as  for 
Question  8. 

12.  The  codes  used  for  the  answers  to  this  question  were  the  same  as 
for  Question  8. 

15.  This  question  was  not  always  clear  to  the  participants.  It  was  not 
clear  to  Che  participants  how  "one  user  at  multiple  locations"  sliould  be 
answered  and  their  answer  was  written  under  "comments".  Tlie  authors 
selected  Answer  15b  as  being  appropriate  and  coded  it  C03. 


13,  16-17.  Questions  13,  16,  and  17  are  related.  Some  participants 
checked  16e  for  an  in-house  project  (Answer  13a  or  13b)  ,  others  wrote 
"none"  under  "Ocher”,  and  other  participants  left  the  question  blank. 
Therefore,  the  authors  elected  to  insert  a  new  answer,  16y,  and  coded  it 
COl  if  Question  16  was  blank  and  the  project  was  for  an  in-house  customer 
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If  Question  16  was  answered  other  than  16c  and  16g  and  Question  17 
was  not,  answer  17y  was  coded  COl.  The  following  code  was  used  for 
answer  17c: 

COM  —  Comparison  of  output  results  against  known  answers. 

18.  The  total  cost  of  the  project  was  reported  in  dollars  according  to 
the  following  method.  The  cost  $  d(l)  d(2)  d(3)  ...  d(N)  can  be 
represented  by  d(l)  d(2)  X  10**R  where  R“N-2  and  was  coded  on  the  listing 
as  d(l)  d(2)  R. 

19.  The  total  cost  of  the  software  was  reported  using  the  same  method 
as  in  Question  18. 

20.  The  project's  beginning  and  ending  was  e.xpressod  in  months  and  years, 
with  the  beginning  month  being  a  number  from  001  to  012  as  answer  a  and 
the  beginning  year  being  the  last  two  digits  of  the  year  as  answer  b; 

the  ending  month  being  a  number  from  001  to  012  as  answer  c  and  tiic 
ending  year  being  the  last  two  digits  of  the  year  as  answer  d.  If  the 
month  was  not  given,  Jun  (006)  was  assumed. 

22.  The  lines  of  source  codes  were  reported  using  the  same  method  as  in 
Question  18. 

2A.  The  size  of  the  data  base  was  reported  in  words  or  bytes  using  the 
same  method  as  in  Question  18. 
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SECTION  2  -  REQUIREMENT  SPECIFICATIONS, 

INPUT  CONDITIONS,  AND  ENVIRONMENT 

25-30.  Questions  25  through  30  are  related.  If  Questions  25  tlirough  30 
generally  indicated  that  requirement  specifications  were  not  prepared  and 
Questions  25  through  30  were  not  answered,  answer  y  was  coded  C03  (or  N/.\ 
for  Question  27) . 

26.  If  Question  25  was  answered  25a  and  Question  26  was  not  answered, 
answer  26c  was  coded  COl.  If  Question  25  was  answered  25b  or  25d  through 
252  and  Question  26  was  not  answered,  answer  26y  was  coded  COl  or  C03. 

29.  If  Question  28  was  answered  28b  or  28y  and  Question  29  was  not 
answered,  answer  29y  was  coded  COl  or  C03. 

30.  Participants  did  not  generally  satisfactorily  answer,  nor  apparently 
understand,  Question  30d.  However,  if  Question  30d  was  answered  the  code 
was  used: 

PDL  —  Programming  Design  Language 

31.  Question  31  did  not  originally  have  a  "Documentation  not  required" 
answer.  Therefore,  there  was  no  way  for  the  participant  to  indicate 
"none".  If  the  participant  was  generally  answering  ail  the  questions  and 
he  left  Question  31  blank,  answer  31y  was  coded  C03. 

32.  Question  32  did  not  originally  have  a  "Customer  did  not  specify" 
answer.  Therefore,  there  was  no  way  for  the  participant  to  indicate 
"none".  If  the  participant  was  generally  answering  ail  the  questions  and 
he  left  Question  32  blank,  answer  32y  was  coded  C03. 

The  codes  to  answer  Question  32ai  and  32a2  was  tlie  same  as  for 
Question  8.  Answers  32a3  and  32a4  were  only  used  if  two  machines  were 
reported . 

The  code  to  answer  Question  32b  was  repotted  using  ti\e  same  method  as 
in  Question  18.  Other  codes  were: 

Percent  number  (%)  —  percent  of  core  used  at  specified  time  in 
development  cycle  (usually  Critical  Design  Review) . 

PCT  —  Percent  used  (unit  unspecified). 

The  code  to  answer  Question  32c  was  indicated  in  time  to  respond. 
Response  time  was  reported  in  seconds  using  the  same  method  as  in 
Question  18,  except  the  second  position  can  be  a  minus  sign  for  negative 
exponential. 

Other  codes  were: 

Percent  Number  (%)  —  percent  of  available  execution  time 
PCT  —  Percent  used  (unit  unspecified) 

RTS  —  Real  time  system 

SRT  —  System  response  or  execution  time  (unit  unspecified). 
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The  following  codes  were  used  Co  answer  Question  3211.  .Vnswer  to 
32d2  was  used  only  if  two  languages  are  reported, 

API  —  AP-101 

ASS  —  Unspecified  Assembler 

CMl  —  CMS'2 

FOR  —  FORTRAN 

HAL  —  HAL/i 

JOV  —  JOVIAL 

PLl  —  PL/1 

TAG  —  TAG  POL/TAG  MOL 

The  following  code  was  used  to  answer  Question  32p. 

Number  —  Number  of  paragraph  modified. 

The  following  codes  were  used  to  answer  Question  32v. 

GGR  —  Critical  component  reliability  only 
HWR  —  Hardware  reliability  specified  only 
MTB  —  Mean  time  between  failures 
SAV  —  Software  system  availability  (up  time) 

SET  —  Software  evaluation  testing 

TSR  —  Total  system  reliability/available. 

The  following  codes  were  used  to  answer  Question  32aa. 

Number  —  Time  in  months 

BOA  —  Basic  ordering  agreement. 

33.  Codes  for  Question  33  can  be  found  under  Type  Four  codes. 
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SECTION  3  -  ?L.\NNING 

34-42,  55.  Questions  34  through  42  and  Question  55  are  related.  If 
Questions  34  through  42  generally  indicated  no  planning  was  done  and  if 
any  Question  34  through  42  was  not  answered,  answer  y  was  coded  COl  if 
36y  or  40y  was  answered,  otherwise  C03  was  used  (.Question  41  was  coded 
N/A) . 

34.  If  Question  55a  was  answered  and  Question  34e  was  not  answered, 
answer  34e  was  coded  C03.  If  Question  55b  was  answered  and  Question  J4a 
was  not  answered,  answer  34a  was  coded  C03.  If  Question  55c  was  answered 
and  Question  34b  was  not  answered,  answer  34b  was  coded  C03. 

35.  Question  35  did  not  originally  have  a  "None"  answer.  Therefore, 

there  was  no  way  for  the  participant  to  indicate  "none  of  the  above".  If 

the  participant  was  generally  answering  all  of  these  questions  and  he 
left  Question  35  blank,  answer  35y  was  coded  C03. 

36.  There  was  a  typo  error  in  answer  36c.  CEM  should  have  been  CPM 
(Critical  Path  Method) . 

37-39.  Questions  37  through  39  are  related.  If  Question  37  was  answered 
37b  through  37y  and  (Question  38  was  not  answered,  answer  33y  was  coded 
COl  or  C03.  In  addition,  the  answer  "lines  of  code"  was  left  out  of  the 
original  possible  choices.  The  authors  felt  chat  had  tlus  answer  been 
provided  more  particpants  would  have  selected  it.  The  answers  that  were 
based  on  size,  i.e, ,  length  of  Che  module,  number  of  programs,  number  of 
instructions,  etc.,  were  lumped  under  "linos  of  code". 

39.  If  Question  37  was  answered  37e,  and  Question  39  was  not  answered, 
answer  39e  was  coded  COl. 

41.  This  question  was  slightly  confusing.  Some  of  the  participants  wore 
not  sure  that  Che  authors  meant  software  only  and  total  time.  However, 
most  of  the  answers  appear  to  be  reasonable. 

42.  Again,  this  question  apparently  was  not  clear  since  all  of  the 
planning  activities  did  not  always  add  up  to  100  percent. 

43-44.  Questions  43  and  44  are  related.  If  Question  43  was  answered  43a 
and  Question  44  was  not  answered,  answer  44a  was  coded  COl. 

45-46.  Questions  45  and  46  are  related.  If  Question  45  was  answered  45y 
and  Question  46  was  not  answered,  answer  46y  was  coded  COl.  This  would 
have  been  a  better  question  had  we  not  specified  only  formal  QA  standards, 
but  used  instead  either  formal  or  documented  QA  standards.  However,  all 
of  the  participants  appeared  to  answer  with  both  in  mind. 

47.  Question  47  did  not  originally  have  a  "None  of  the  above"  answer; 
therefore,  there  was  no  way  for  cl»e  participant  to  indicate  "none".  If 
the  participant  was  generally  answering  all  the  questions  and  he  left 
Question  47  blank,  answer  47y  was  coded  C03. 
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31,  48.  Questions  31  and  48  are  related.  Question  43  did  not  originally 
have  a  "None"  answer,  therefore,  there  was  no  way  for  the  partiopant  to 
indicate  "none".  If  the  participant  was  generally  answering  all  tiie 
questions  and  he  left  Question  48  blank,  answer  48y  was  coded  C03.  Also, 
Question  48  was  so  similar  to  Question  31  Chat  if  the  participant  did  not 
answer  Question  48  but  he  did  answer  Question  31,  the  same  answers  were 
used . 

46-48.  Questions  46  through  48  are  related.  If  Questions  4d,  47,  and 
48  were  answered  y  and  Question  49  was  not  answered,  answer  49y  was 
coded  C03. 

50.  Question  50  did  not  originally  have  a  "None"  answer;  tiierefore, 
there  was  no  way  for  the  participant  to  indicate  "none".  If  Che 
participant  was  generally  answering  all  questions  and  he  left  Question 
50  blank,  answer  50y  was  coded  C03. 

51-52.  Codes  for  Questions  51  and  52  can  be  found  under  Type  Four  codes. 
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SECTION  4  -  ORGANIZATION 

53.  This  question  was  not  always  answered  properly  (e.g.,  many  people 
filled  in  "project  manager"  under  "other").  By  referring  to  later 
questions,  it  was  apparent  thay  they  meant  one  of  the  multiple-choice 
answers,  and  it  was  changed  by  Che  authors  and  coded  C03. 

55,  34.  Question  55  is  related  to  Question  34.  If  Question  34a  was 
answered  and  Question  55b  was  not,  answer  55b  was  coded  C03.  If  Question 
34e  was  answered  and  Question  55a  was  not,  answer  55a  was  coded  C03. 
However,  if  Question  55  was  answered  the  following  code  was  used  to 
answer  Question  55b: 

ACB  —  Administration  Control  Board. 

56.  On  several  occasions  the  participant  answered  either  Question  56a  or 
b  and  one  of  Questions  56c  through  f.  This  was  frequently  redundant  and 
the  answers  to  Questions  56c  through  56f  were  eliminated. 

58.  This  question  is  related  to  Question  22,  Part  One.  If  this  question 
was  incompletely  answered.  Part  One  answers  and  participants  comments 
were  used  to  answer  Question  58  (coded  C03) .  Also,  through  hindsight  and 
written  comments,  two  more  questions  were  added  which  appeared  to  make 
Che  answer  sec  more  complete. 

59.  Again,  this  question  was  not  properly  answered.  The  participants 
apparently  did  not  understand  the  difference  between  authorized  and 
assigned.  However,  if  Question  78A  was  answered  and  Question  59  was  not, 
the  sum  of  answers  to  78A  was  inserted. 

60-65.  Questions  61  through  65  are  related  to  Question  60.  If  Question 
60b  was  answered  and  Question  61  was  not,  answer  61y  was  coded  COl.  This 
question  could  have  been  improved  by  including  engineers  with  analysts. 

63.  If  Question  6id  was  considered  to  be  a  "no"  or  Question  60b  was 
answered  and  Question  63  was  not  answered,  answer  63y  was  coded  COl. 

64.  If  Question  120  was  answered  properly  (and  it  often  was  not)  and  if 
Question  64  was  not  answered,  the  correct  answer  could  be  derived. 

65.  If  Question  60b  was  answered  and  Question  65  was  not,  answer  65y  was 
coded  COl.  There  was  a  variety  of  answers  written  into  this  question. 
Apparently  the  title  of  "team  chief"  is  not  constant  within  industry.  The 
more  common  ones  that  appeared  to  be  similar  were  lumped  under  answer  65d 
(a  new  answer) . 

66.  There  were  almost  no  answers  to  this  question.  Apparently  project 
managers  do  not  make  use  of  some  of  these  exotic  Cities.  The  only  one 
constantly  written  in  was  "software  engineering".  In  addition,  since 
Question  66  did  not  originally  have  a  "None  of  the  above"  answer  there  was 
no  way  for  the  participant  to  indicate  "none".  If  the  participant  was 
generally  answering  all  Che  questions  and  he  left  Question  66  blank, 
answer  66y  was  coded  CO  3. 

67-68.  Codes  for  Questions  67  and  68  can  be  found  under  Type  Four  codes. 
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SECTION  5  -  STAFFING 

68.  Question  68  was  duplicated.  To  avoid  having  to  renumber  all  the 
questions  from  68  on,  the  second  Question  68  was  renumbered  69f. 

69-77.  Questions  69  through  77,  and  86  are  dependent  on  Question  57.  If 
Question  57  was  answered  57y  (no  project  manager)  all  y  answers  in 
Question  69  through  77  were  coded  COl  (N/A  for  Question  73). 

If  Question  69f  was  answered  the  following  codes  were  used:  (These 
same  codes  were  used  for  Questions  93,  94,  105h,  109,  118,  and  120). 

Senior  Corporate  Officers 

VPC  —  Vice  President,  Data  Processing 

VPE  —  Vice  President,  Engineering/Functional  Area 

VPG  —  President  or  Vice  President,  General 

Senior  Management 

MCP  —  Senior  Manager,  ADP 

MEN  —  Senior  Manager,  Engineering/Functional  Area 

MGR  —  Senior  Manager,  General  (includes  Division  Manager) 

MPA  —  Assistant/Deputy  Program  Director 

MPD  —  Senior  Program  Director  (as  opposed  to  Project  Manager) 
Project  Management 

PEN  —  Project  Engineer 

PMA  —  Assistant  Project  Manager,  Deputy  Project  Manager 
PMC  —  Project  Manager,  ADP/Coraputer 
PME  —  Project  Manager,  Engineering/Analyst 
PMR  —  Project  Manager 
PMS  —  Project  Manager,  Subsystem 
PMW  —  Project  Manager /Software 
PTD  —  Technical  Director/Manager 
First  Line  Supervisor 

FAP  —  Applications  Supervisor 
FEN  —  Engineering  Supervisor 
FFS  —  Fiscal  Supervisor 

FLS  —  First  Line  Supervisors,  General  (includes  group  leaders,  task, 
leader,  section  head,  technical  leader,  manager,  supervisor, 
head,  etc.) 


FOS  —  Operations  Supervisor 
FPC  —  Project  Control  Supervisor 
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FSA  —  Application  Software  Supervisor 

FSD  —  Software  Development  Supervisor 

FSE  —  System  Engineer  Supervisor 

FSQ  —  Software  Quality  Assurance  Manager 

FSR  —  Software  Requirements  Supervisor 

FSS  —  System  Software  Supervisor 

FSW  —  Software  Supervisor 

FSY  —  Systems  Supervisor 

FTC  --  Team  Chief 

FTE  —  Technical  Supervisor 

FTI  —  Test  and  Integration  Supervisor 

FTM  —  Test  Supervisor 

FVS  —  Verification  Supervisor 

Lead  ADP  Personnel  (includes  Senior,  Lead,  Senior  Project,  Chief,  etc., 

ADP  personnel) . 

LPA  —  Lead/Senior  Programmer/Analyst 
LSA  —  Lead/Senior  .Analyst 
LSP  —  Lead/Senior  Programmer 

Lead  Engineer/Functional  Personnel  (includes  Senior,  Lead,  Senior  Project, 
Chief,  etc.,  Engineering/Functional  personnel). 

LSE  —  Lead/Senior  Engineer 

LSW  —  Lead  Software  Engineer 

ADP  Personnel 

CAN  —  Analyst 

CDA  —  Data  Base  Analyst 

CDV  —  Software  Developer 

CHP  —  Chief  Programmer 

CMS  —  Software  Configuration  Management 

COP  —  Computer  operations 

CPA  —  Programmer /Aanlyst 

CPJ  —  Junior  Programmer/Programmer  Aid 

CPR  —  Programmer 

Engineer/Functional  Personnel 


EAN  —  Engineering  Analyst 
ECH  —  Computer  Hardware  Enginer 
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EHW  —  Hardware  Engineer 
EIG  —  Integration  Engineer 
ENG  —  Engineer/Functional/Designer 
ENJ  —  Junior/Associate  Engineer  -  Engineering  Aid 
ENS  —  System  Engineer 
ENT  —  Test  Engineer 
ESW  —  Software  Engineer 
Supporting  Staff 

SAC  —  Accountant 

SAD  —  Administration 

SAS  —  Staff  Assistant 

SEC  —  Secretary/Typist 

SLI  —  Support  Librarian 

SMG  —  Support  Manager/Supervisor 

SOP  —  Functional  Operation 

SPG  —  Project  Control 

STS  —  Testing 

General  (Unspecified  Personnel) 

WMT  —  Member  Technical  Staff 
WOR  —  Worker,  Individual,  Staff 
Other  Positions 

OCU  —  Customer 
OMA  —  Mathematician 
OTH  —  Other 

72.  The  following  codes  were  used  to  answer  this  question: 

001  —  High 
002  —  Average 
003  —  Low 

75.  Since  Question  75  did  not  oroginally  have  "None"  for  an  answer  there 
was  no  way  for  the  participant  to  answer  "none".  IE  tlie  participant  was 
generally  answering  all  the  quesC.ions  in  this  section  and  he  did  not 
answer  Question  75,  answer  75y  was  coded  C03.  However,  if  he  did  answer, 
the  codes  used  to  answer  Question  75f  are  the  same  as  used  in  Question  32d. 
Answer  75E  was  divided  into  two  answers,  75fi  and  75f2,  to  accommodate  two 
languages  and  used  the  same  codes  as  Question  32d. 
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lb-11.  Questions  76  and  77  are  related.  If  Question  76a  or  b  was 
answered  and  Question  77  was  not  answered,  answer  77y  was  coded  COl.  In 
addition,  two  new  answers  were  provided  for  this  question — physics  and 
science  (an  oversight  on  the  part  of  the  authors) . 

78.  This  question  confused  many  of  the  surveyees.  However,  the  answers 
are  included  in  hopes  they  will  provide  benefit  to  somebody. 

80.  If  Question  78f  was  answered  by  a  "none"  and  Question  80  was  not 
answered,  answer  80y  was  coded  COl. 

81.  If  Question  65a  was  answered  by  a  "none"  and  Question  31  was  not, 
answer  81y  was  coded  COl. 

83-84.  The  answers  to  both  of  these  were  derived  from  the  written 
answers  provided. 

85.  The  following  codes  were  used  in  answering  this  question: 

BBB  —  Both  within  and  outside  the  project  resources 

000  —  Outside  the  project  resources 
WWW  —  Within  the  project  resources. 

86.  If  Question  57y  was  answered  and  Question  86  was  not,  answer  86y 
was  coded  COl. 

90.  Codes  for  Question  90  can  be  found  under  Type  Four  codes. 
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91-94.  Questions  92  through  94  were  dependent  on  Question  9i.  If 
Question  91y  was  answered,  those  Questions  92  through  94  which  were  not 
answered  were  answered  with  a  y  and  coded  COl. 

91.  Again,  as  in  Question  36,  "CEM"  was  corrected  to  "CPM"  and  ocher 
systems  were  added.  Several  of  chose  chat  were  graphic  in  nature  were 
lumped  under  Che  general  type  "graphics". 

92.  If  Question  35c  was  answered  "negative"  and  92  was  not  answered, 
answer  92b  was  coded  CO 3. 

93-94.  If  Question  91y  was  answered  and  Question  93  was  not,  answer  93y 
was  coded  COl.  If  Question  91y  was  answered  and  Question  94  was  not, 
answer  94y  was  coded  COl. 

If,  however.  Question  93  and/or  94  were  answered  the  codes  used  in 
Question  69f  were  used. 

99.  If  Questions  60,  61,  or  65  indicated  chat  teams  were  not  used  and 
if  Question  99  was  not  answered,  answer  99y  was  coded  COl.  The  following 
codes  were  used  in  answering  this  question: 

001  —  Always 

002  —  Most  of  the  time 

003  —  About  one-half  of  the  time 

004  —  Less  chan  one-half  of  the  time 

005  —  Seldom 

006  —  Never 

99-101.  If  Questions  99d  or  99e  were  answered  "never"  and  Question  100 
or  101  was  not  answered,  answer  lOOy  and  lOly  were  coded  C03. 

102-103.  Questions  102  and  103  are  identical  to  Questions  20  and  21, 

Part  One.  If  Question  20  was  answered  "no"  and  Questions  102  and  103 
were  not  answered,  answers  102b  and  103y  were  coded  COl. 

104-105.  Questions  104  through  105  are  related.  If  Question  104  is 
answered  104y,  and  105  is  not  answered,  answer  105y  was  coded  COl. 

105.  Question  105  was  originally  incorrectly  numbered.  Therefore,  it  was 
renumbered  105a  through  105h,  and  105y.  If  Question  i05h  was  answered 
Che  codes  used  in  Question  69f  were  used. 

106,  108-109.  Questions  106.  108,  .ind  109  are  rcl.itcd.  If  (Juestion  106 
is  answered  "no"  and  Question  108  and  109  arc  not  answered.  Question  10Kb 
was  answered  COl  and  Question  109  was  answered  N/A. 

106.  There  were  almost  no  answers  given  to  Question  106a;  however,  if 
answered,  Che  following  codes  are  used  in  answering  this  question; 

CCS  —  Configuration  Control 

CUS  —  Customer  supplied 
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HAC  —  HAC  syscetn 

IHD  —  In-house  developed 

LIP  ~  LIP  SVC 

MAN  —  Manual 

SDM  —  Software  Development  Manager 

SKD  ~  SKD 

483  ~  MIL-STD-483 

109.  The  codes  used  in  answering  this  question  were  the  same  as  for 
Question  69f. 

110-114.  The  Questions  110  through  114  are  related.  If  any  of  the 
questions  are  answered  with  a  y,  then  those  that  are  not  answered  will 
have  y  coded  with  a  COl. 

115.  Codes  for  Question  113  can  be  found  under  Type  Four  codes. 
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SECTION  7  -  DIRECTING/ MONITORING 

118.  IE  QuesCion  118  was  noC  answered  someCime  an  answer  could  be  derived 
from  elsewhere  In  the  survey.  If  Quescion  118  was  answered  the  codes 
used  are  the  same  as  for  Question  69f. 

119.  This  question  was  completely  confusing  to  many  people  who  apparently 
did  not  know  a  definition  of  the  term  "span  of  control".  Span  of  control 
was  meant  by  the  authors  to  be  the  number  of  people  that  were  under  the 
direct  supervision  of  the  project  manager.  From  the  large  numbers  chat 
appeared,  apparently  most  people  answered  the  total  number  of  people  chat 
were  under  the  project  manager. 

120.  The  codes  used  for  answering  120A  and  120B  are  the  same  as  for 
Quescion  69f.  If  Quescion  120g  was  not  answered  it  could  sometimes  be 
derived  from  answers  to  120A,  B  and  C. 

121.  Quescion  121  is  dependent  on  Questions  104  and  ILO.  If  Question  104 
and  110  answered  i04y  and/or  liOy  and  Question  121  was  not  answered, 
answer  i21y  was  coded  C03. 

123-124.  Questions  123  and  124  are  related.  If  the  answer  to  Question 
123  was  "no"  and  124  was  not  answered,  answer  124y  was  coded  COl .  If 
Question  124  was  answered  124a,  b  or  c  and  Question  123  was  not  answered, 
answer  123Ba  was  coded  C03. 

123.  Answer  123c  had  a  typo.  It  was  corrected  from  "moderation"  to 
"motivation" . 

127.  Codes  for  Quescion  12/  can  be  found  under  Type  Four  codes. 
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SECTION  8  -  DELIVERABLES  AND  SUCCESSES 

In  general.  Section  3  lacked  one  poinc  of  claricv.  Ic  was 
authors'  intent  to  talk,  only  about  software  deliverables  and  successes. 
However,  the  word  software  was  not  always  emphas ized ,  and  we  believe 
that  some  of  the  answers  involved  the  entire  project,  whicii  could  have 
been  partly  hardware  and  partly  software.  ViTnere  the  project  was  all 
software,  of  course,  the  answers  applied  to  software.  In  those  incidences 
where  software  was  the  pacing  factor,  then  we  are  sure  the  answers  also 
applied  only  to  software.  Where  the  pacing  factor  was  hardware,  you 
could  be  led  to  believe  that  the  problems  were  associated  with  hardware 
and  not  software.  The  authors  leave  it  up  to  the  reader's  discretion  to 
evaluate  exactly  what  the  answers  mean. 

123-149.  Questions  128  through  149,  153  and  134  are  related.  i: 

Question  128a,  128b  or  128y  was  answered  (or  some  comment  to  the  fact 
there  was  insufficient  data  to  complete  this  section)  and  Questions  i29 
through  149,  153  and/or  154  were  not  answered,  answers  y  were  coded  COl 
(Questions  132  and  135  were  coded  "N/A") . 

129-133.  Questions  129  through  133  are  related.  If  'I'uestion  129a  was 
answered  and  Questions  130  through  133  were  not  answered,  answer  y  was 
coded  COl,  (Question  130  was  coded  "N/A".) 

129.  Answers  to  Question  129b  were  divided  into  i29bl  and  129b2,  and 
answers  to  Question  129c  were  divided  into  i29cl  and  i29c2.  Answers  bl 
and  cl  are  reported  in  number  of  months;  answers  b2  and  c2  are  given  in 
percent . 

130.  If  Question  i30a  was  answered  "negative"  and  Question  132  was  not 
answered,  answer  132  was  coded  "N/.A". 

131.  This  question  has  been  deleted  from  this  modified  version  of  Part 
Two  of  the  questionnaire.  It  was  meant  to  have  been  deleted  during  the 
last  typing  because  it  duplicated  Question  129,  but  it  was  left  in  by 
error . 

134-137.  Questions  134  through  137  are  related.  If  Question  134a  was 
answered  and  Questions  133  through  137  were  not  naswered,  answer  y  was 
coded  COl  (Question  136  was  coded  "N/A") . 

134.  Answers  to  Question  134b  were  divided  into  134bl  and  134b2,  and 
answers  to  Question  134c  were  divided  into  134cl  and  134c2.  Answers  bl 
and  cl  are  reported  in  dollars  using  the  same  method  as  in  Question  18; 
answers  b2  and  c2  are  given  in  percent. 

135.  If  Question  135a  was  answered  "negative"  and  Question  136  was  not 
answered,  answer  136  was  coded  "N/A". 

138-141.  Questions  138  through  141  are  related.  If  Question  138y  was 
answered,  and  Questions  139  through  141  were  not,  answer  y  was  coded  COl. 

141.  If  Question  138a  was  answered  and  Question  141  was  not,  answer  141d 
was  coded  COl. 

142-144.  Questions  142  through  144  are  related.  If  Question  142y  was 
answered  and  Questions  143  and  144  were  not,  answer  y  was  coded  COl. 
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143.  If  Question  142a  was  answered  and  143  was  not,  answer  i43d  was 
coded  COl. 

145-147.  Questions  145  through  147  are  related.  If  Question  145y  was 
answered  and  Questions  146  and  147  were  not,  answer  y  was  coded  COl. 

145.  If  Question  145a  was  answered  and  146  was  not,  answer  146d  was 
coded  COl. 

146.  As  stated,  those  questions  that  were  originally  narrative  have 
been  turned  into  multiple  choice  questions  by  taking  the  given  answers 
and  making  them  into  logical  groupings.  There  were  too  few  answers  to 
Question  146a  to  do  this;  therefore,  it  is  left  in  its  original  narrative 
form. 

155-157.  Questions  155  through  157  are  related.  If  Questions  155 
through  157  indicate  that  on-line  programming  was  not  used  in  the  develop¬ 
ment  of  this  project  and  some  or  all  of  these  questions  were  not 
answered,  answers  y  were  coded  COl. 

159.  Codes  for  Question  159  can  be  found  under  Type  Four  codes. 

PART  THREE 

Questions  1,  2,  3  and  4,  and  25  of  Part  Three  are  added  to  this 
report  as  Questions  160,  161,  162,  163,  and  164. 

161-162.  Questions  161  and  162  are  related.  If  Question  16ia  was 
answered  and  Question  162  was  not,  answer  i62y  was  coded  COl.  If 
Question  161c  was  answered  and  Question  162  was  not,  answer  162g  was 
coded  COl, 

164.  Codes  for  Question  164  can  be  found  under  Type  Four  codes. 
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TYPE  FOUR  CODES 


This  section  reflects  the  codes  used  to  answer  narrative  or  lessons 
learned  type  questions.  With  the  exception  of  Question  164  all  codes 
are  listed  by  type  rather  than  question. 

Codes  for  Question  164. 

AOl  Software  engineering  management  is  not  different  from  Sij  or 
prototype  development  of  hardware. 

A02  It  is  an  error  to  compare  software  development  with  the 
production  phase  of  hardware. 

A03  Many  questions  are  philosophic. 

A04  Software  engineering  is  new. 

A05  Many  approaches  to  software  development  management  are  valid. 

A06  Initial  higher  cost  at  the  beginning  of  a  project  caused  by 

using  software  engineering  techniques  will  be  offset  by  savings 
later  on. 

A07  Software  development  is  a  tough  management  problem. 

A08  We  do  not  know  if  the  initial  higher  cost  of  software  engine¬ 
ering  techniques  will  be  offset  by  savings  later  on. 

A09  Case  studies  of  software  development  are  required. 

AlO  Cost  and  scheduling  of  a  project  are  a  problem. 

All  Survey  was  too  theoretical  (orderly  progress  from  beginning  to 
end)  for  our  project. 

A12  Requirement  specf iciations  required  extensive  modification 
before  development. 

A13  We  should  expect  some  of  the  "art"  of  software  development  to 
always  remain  with  us  and  accept  it. 

A14  Survey  was  exhaustive  and  exhausting. 

A15  Good  luck. 

Alb  Too  much  emphasis  upon  quantification  of  subjective  functions. 

A17  Most  important  function  is  visibility  of  progress  and  automateil 
procedures . 

Ai8  Hope  you  get  many  returns. 

A19  I  hope  you  do  not  find  my  answers  too  difficult  to  underst.and. 

AlO  Many  good  ideas  in  Questionnaire,  good  job. 

A21  You  should  follow  on  answers  with  telephone  interviews. 

A22  Survey  was  redundant  and  could  be  shortened. 

A23  Good  judgement  and  experience  is  more  important  than  cook  book 
techniques  and  Cools. 
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A24  Survey  was  not  Coo  applicable  to  be  answered  by  customer. 

A25  My  answers  to  these  questions  came  from  my  experience  in 
developing  command  and  control  systems. 

A26  Data  should  be  collected  by  interview. 

A27  Separating  software  development  management  from  iiardwarc 
development  management  in  this  survey  is  a  mistake. 


How  the  narrative  answers  were  coded.  The  first  element  of  the  code 
(one  or  two  letters)  is: 

R  —  Requirement  function  (this  is  a  sub-sec  of  planning) 

P  —  Planning  function 

X  —  Design,  Code  and  Test  function  (this  is  a  sub-function  of 
planning) 

0  —  Organizing  function 

S  —  Staffing  function 

ST  — •  Training 

C  —  Controlling  function 

CV  —  Reviews 

CW  —  Walk-throughs 

CS  —  Software  standards 

CM  —  Configuration  management 

CT  —  Test  and  verification 

C  —  Directing  function 

The  second  element  (when  used)  is  a  number  in  order  to  distinguish 
between  otherwise  identical  codes,  except  "R”  which  is  used  to  designate 
the  surveyee  suggested  R&D  is  necessary. 


Requirements 

R1  Software  requirement  specifications  (baseline)  should  iiavc  the 
following  general  attributes. 

a.  Accomplish  its  intended  function  of  specifying  the  requirements. 

b.  A  (more)  formal  project  phase  (function) 

c.  Based  on  a  (thorough/careful)  analysis 

d.  Correct 

e.  Useable  by  the  software  developer 

f.  (more)  Complete 

g.  Clear,  readable  and  understandable 

h.  Unambiguous  and  consistent 

i.  (more)  Specific  and  detailed 

j.  Delivered  on  time  (before  beginning  softwar:  design) 

k.  Reviewed  prior  to  use 

l.  (more)  Emphasized  throughout  the  project 

m.  Testable 

n.  Compatible  with  other  functions  (hardware) 

0.  ControLled  and/or  baselined 

р.  Any  attributes  or  method  will  be  acceptable 

y.  None  required. 

R2  Software  requirement  specifications  should  not  (be/have) ; 

a.  Include  detail  design 

b.  Only  describe  the  process 

R3  Software  requirement  specifications  should  contain  the  following: 

a.  A  description  of  the  user's  operation 

b.  Test  requirements 

с.  Detailed  test  plan 

Rd  Software  requirement  specifications  should  be  developed  jointly  (or 
involve)  personnel  assigned  to  the  following  functions  (this  is  also 
part  of  the  organization  function); 

a.  Contractor 

b.  Customer 


c.  User 

d.  Operator 

e.  Programmer 

f.  System  engineer 

g.  Software  engineer 
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R7  Software  requirement  specif ications  should  be  developed  through  the 
use  of: 

a.  Top-down  techniques 

b.  Structured  techniques 

c.  Hierarchical  techniques 

d.  Incremental  (phased)  techniques 

e.  Modular  techniques 

f.  Simulation  techniques 

g.  (more)  Automated  tools 

R9  Miscellaneous  lessons  learned  and/or  comments  concerning  software 
requirements  specifications. 

a.  Agree  on  specifications  between  customer  and  developer. 

b.  Make  sure  system  design  is  feasible  before  beginning  software 
design. 

c.  If  you  want  the  software  to  be  reliable,  maintainable,  etc., 
put  it  in  the  contract. 

d.  More  emphasis  on  reliability,  maintainability  and  useability. 

e.  Many  people  do  not  know  how  to  write  requirement  specifications. 

f.  MIL-STD-483  needs  updating/ improvement. 

RR  R&D  in  software  requirement  techniques  should  include: 

a.  Developing  a  formal  requirement  methodology. 

b.  How  to  do  requirement  specifications  development  through 
simulation. 

c.  How  to  analyze  requirement  specifications. 
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Planning 

P’  The  planning  function  should  have  the  following  general  attributes 

a.  Accomplish  its  intended  function  (correctly  and  cost 
effectively) . 

b.  Based  on  a  (thorough/careful)  analysis. 

c.  (more)  Specific  and  detailed. 

d.  Delivered  on  time  (early  in  project). 

e.  (more)  Fully  documented. 

f.  Reviewed  prior  to  use. 

g.  Reviewed  throughout  the  project. 

h.  (more)  Emphasized  throughout  the  project, 

y.  None  required. 

P3  Planning  should  include  (be  improved  in)  the  areas  of; 

a.  A  development  plan  and  procedures. 

b.  Accurate  cost  and  scheduling. 

c.  Formalized  and  detailed  testing  planning. 

d.  Training  plan. 

e.  A  fall  back  position  if  problems  develop. 

f.  Prioritizing  work  load. 

g.  Staff  functions. 

P4  Planning  and  the  planning  function  should  be  developed  jointly  (or 
involve)  personnel  asfiigned  to  the  following  functions  (this  is  also 
part  of  the  organization  function) : 

a.  Contractor. 

b.  Customer. 

c.  Project  personnel. 

d.  Software  analyst. 

e.  Programmer. 

P5  The  project  and/or  project  manager  should  be  provided  with  the 
following: 

a.  More  resources. 

b.  Better  availability  of  (additional)  supporting  resources. 

c.  Better  availability  of  (additional)  computer  support. 

d.  Better  availability  of  development  and  test  tools  and  support 
software. 

e.  Better  availability  of  (additional)  administrative  support. 

f.  Adequate  direction  and  guidance. 
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g.  A  means  of  determining  computational  efficiency. 

h.  Access  to  existing  in-house  expertise. 

i.  .A  formal  support  library  system. 

P8  In  planning  for  a  software  development  project,  more  time  and  effort 
should  be  allowed  for  the  following  project  functions; 

a.  The  requirement  specification  phase. 

b.  The  planning  phase. 

c.  The  software  development  phase. 

d.  Cost  and  scheduling  phase. 

e.  The  design  phase. 

f.  The  programming  phase. 

g.  The  test  phase. 

h.  Reviews. 

i.  Documentation 

j.  All  phases  of  the  development. 

P9  Miscellaneous  lessons  learned  and/or  comments  concerning  planning: 

a.  Do  not  allow  the  planning  function  to  be  pre-empted  by  short 
range  tasks. 

b.  Determine,  in  advance,  if  the  development  facility  will  support 
the  project. 

c.  Do  not  use  "seat  of  the  pants"  planning. 

d.  The  first  time  a  system  is  built  is  always  a  problem. 

e.  All  resources  should  be  in  close  proximity. 

f.  Use  vendor  supplied  OS. 

g.  Do  not  let  customer  change  schedule  and  software  design  with¬ 
out  contractor  agreement. 

h.  Politics  plays  a  more  important  role  than  technology. 

i.  Have  more  than  a  few  "super  stars"  on  the  team  if  you  expect 
success . 

j.  Estimate  high;  expect  the  worse. 

k.  Scheduling  around  holidays  caused  delays. 

PR  R&D  in  the  planning  function  should  include: 

a.  Improvements  in  sizing,  cost  and  estimating  methods  (accuracy) . 

b.  A  better  method  to  assess  impact  of  changes  on  cost  and  schedules. 

c.  Determining  (research  in)  software  life-cycle  costs. 


XI  The  software  development  function  should  have  the  following  general 
attributes : 


a.  A  (more)  formal  project  phase. 

b.  Correct. 

c.  (more)  Specific  and  detailed. 

d.  Delivered  on  time. 

e.  (more)  Fully  documented. 

f.  Controlled  and/or  baselined. 

X4  The  software  should  be  developed  Jointly  by  (or  involve)  personnel 
assigned  to  the  following  functions  (this  is  also  part  of  the  organization 
function) : 

a.  User. 

X7  Software  should  be  developed  through  the  use  of; 

a.  Project  management  system. 

b.  Top-down  techniques. 

c.  Structured  (development)  techniques. 

d.  Modular  techniques. 

e.  Appropriate  (HOL)  language  techniques. 

X9  Miscellaneous  lessons  learned  and/or  comments  concerning  software 
development . 

a.  Consider  all  past  experiences  in  planning  a  new  project. 

b.  Have  all  unproven  software  technologies  accepted  by  a  review 
team  prior  to  use  on  project. 

c.  Software  development  more  a  management  problem  chan  a 
technical  problem. 

d.  More  emphasis  on  automated  software  design  tools. 

e.  Use  industry  standard  software  design  techniques. 

f.  System  designers  should  have  more  knowledge  of  software  during 
early  design  stage. 

g.  Documentation  and  flow  charting  are  very  e.-'-ponsivc. 

li.  Average  software  designer  fails  to  see  use  fur  d  i  sc  i|)  I  iue  until 
job  is  near  done. 

i.  Project  manager's  approach  should  be  validated. 

j.  Team  members  should  select  development  methods  used. 

k.  Use  a  common  language  where  practical. 
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i.  Use  seif-annocacing  source  Lises. 

m.  Element  requirement  foe  flow  charts  (prior  to  code  completion) 

n.  More  des-k.  checking  is  required. 

o.  Complete  detail  software  design  before  beginning  programming. 

p.  Do  not  freeze  design  before  completion  of  coding. 

XR  R5.D  in  software  development  techniques  should  include: 

a.  More  design  of  (automated)  design  tools  and  aids. 

b.  Surveying  and  evaluating  available  software  development  tools 
and  techniques. 

c.  Develop  (and  disseminate)  design  validation  techniques. 

d.  Publish  software  design  techniques  standards. 

e.  Develop  a  system  design  language. 

f.  Research  into  software  reliability;  error  types  and  solutions 
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Organizing 


01  The  organizational  structure  (rules  and  authority  lines)  should 
accomplish  the  following: 

a.  Be  defined. 

b.  Be  enforced. 

c.  Define  the  authority  line  (better) . 

d.  Define  the  responsibility  (better). 

e.  Define  a  single,  leadership  function. 

f.  Define  relationship  between  administration  and  technical 
personnel . 

02  The  role  of  .-iDP  in  the  organization  is  to  include: 

a.  Combine  ail  programmers  and  analysts  into  one  group. 

b.  Have  the  programmer  and  analyst  report  to  the  ADP  manager. 

c.  Assign  software  specialist  to  the  senior  manager  staff. 

d.  Have  the  ADP  function  report  higher  up  in  the  projec". 

03  Require  (or  permit)  the  following  organization  structure  and/or 
responsibilities : 

a.  Software  to  be  responsible  to  overall  system  requirements. 

b.  Hardware  to  be  responsible  to  overall  system  requirements. 

c.  Program  manager  (with  the  authority  to  do  the  job). 

d  (more)  Authority  to  the  technical  subordinates  (less  central 
authority  to  project  manager) . 

e.  Software  test  team  not  under  the  project  manager. 

f.  Software  te;  ng  under  software  project  manager. 

g.  A  (large)  technical  staff. 

h.  Senior  management  to  participate  in  project. 

04  Miscellaneous  lessons  learned  and/or  comments  concerning  organiz¬ 
ation  structure: 

a.  Interface  at  detail  level  with  customer  i  costly  and  time- 
consuming  . 

b.  Interface  with  IV&V  contractor  i.s  costly  and  l  Lmc— con.suming . 

c.  Should  not  allow  intermediate  agencies  between  cur tomcr/user 
and  the  designer /programmer . 

d.  Should  not  allow  involvement  of  organizations  and/or 
personnel  who  cannot  contribute  to  the  software  development. 

05  The  organization  structure  (type)  that  is  best  suited  for  a  soft¬ 
ware  engineering  project  is: 

a.  A  formal  organization. 
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b.  Taiiored/adapced  Co  the  job. 

c.  Best  suited  to  company  environment. 

d.  Depends  on  size  of  project. 

e.  A  functional  organization. 

f.  A  line  organization. 

g.  A  project  (task)  organization. 

h.  A  matrix  organization. 

i.  Organized  under  a  project  manager  (with  full  authority) . 

j.  Organized  under  a  manager  with  (strong)  teciinical  respocsi 
bilicies . 

k.  Organized  under  a  manager  with  (complete)  administrative 
responsibilities . 

l.  Organized  under  an  ADP  organizational  function. 

m.  Not  organized  under  an  .ADP  organizational  function. 

n.  Doesn't  matter  what  type  of  organization. 

y.  The  project  was  not  organized. 

06  Ocher  special  organization  structures  (types)  used  or  should  be 
used  are: 

a.  Special  planning  team. 

b.  Integrated  programmer-analyst  teams. 

c.  Project  teams. 

d.  A  standing  review  team. 

e.  Chief  programmer  teams. 

f.  Requirements  analyst  team. 

g.  Small  design  and/or  programming  teams. 

h.  (separate)  Interface  team. 

1.  (separate)  Test  teams. 

j.  (separata)  Documentation  team. 

07  The  software  development  project  (teams)  should  have  as  members 

a.  Customers. 

b.  Managers. 

c.  Functional  analysts. 

d.  Programmers. 

e.  Junior  programmers/programmer  aids. 

f.  Support  librarian. 

g.  Lead  (chief)  programmers. 


h.  Back-up  programmers. 

i.  Hardware  vendors. 

09  Miscellaneous  lessons  learned  and/or  commencs  concerning  project 
organization  types  are; 

a.  The  project  organization  should  have  a  permanently  assigned 
staff . 

b.  The  matrix  organization  enables  people  to  be  removed  for  shor 
periods  of  time  without  hurting  project. 

c.  The  matrix  organization  improved  the  overall  training  program 
OR  RSiD  in  organizational  functions  should  include: 

a.  Research  <^ir  planning  for  an  organization. 

b.  ResearcH  for  a  testing 'organization. 

c.  How  many  support,  librarians  are  required  on  a  project. 
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Staffing 

51  The 

a. 

b. 

c . 

d. 

e. 

f . 

8- 

h. 

i. 

52  The 

a. 

b. 

c . 

d. 

e. 


S5  The 

a. 

b. 

c. 

d. 

ST  The 

a. 

b. 

c . 

d. 

e . 

f . 

g- 

h. 

i. 


staffing  function  should: 

Provide  an  adequate,  qualified  staff. 

Allow  for  early  moving  of  personnel. 

Provide  for  an  experience  and  skill  mix. 

Identify  quality  of  persons  early. 

Have  "open  ended"  training  programs  as  "personnel  buffers". 

Assign  stronger  technical  personnel  to  head  teams. 

Apply  appropriate  talent  ot  the  appropriate  jobs. 

Use  different  level  of  programmers  for  different  aspects  of 
job . 

Have  (more)  permanently  assigned  staff, 
staffing  function  should  not  (be/have)  : 

Remove  personnel  to  put  out  brush  fires. 

Build  up  the  staff  too  fast. 

Have  too  many  (high-level)  people  during  initial  design. 

Select  software  test  team  members  from  programmers  who 
designed  programs. 

Have  a  general  lay-off  (reduction-in-force)  just  prior  to 
starting  project. 

project  manager  should  (be/have): 

Manager ially  and  technically  qualified  for  the  job. 

Good . 

Experience  in  progratnroing  techniques. 

Experience  in  management  techniques, 
training  function  should  (be/have) : 

Provide  adequate  training. 

A  (more)  formalized,  staffed  training  program  (function). 
Better. 

Screen  personnel  for  skills  to  eliminate  on-tlie-Job  training. 
Identify  training  requirements  early. 

Provide  training  in  new  design  technologies. 

Provide  training  in  basic  programming  and  documentation. 
Provide  training  in  application  function. 

Provide  training  in  documentation  principals. 
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j.  Provide  craining  in  software  acquisition. 

k.  Provide  training  in  management. 

l.  Provide  training  in  hardware  used. 

m.  A  better  training  facility. 

SR  R&D  in  staffing  should  include: 

a.  (Better)  methods  of  evaluating  background/experience  (vs 
project  requirement). 

b.  A  study  of  what  factors  influence  proper  mixture  of  skills. 

c.  Different  skills  and  specializations  needed  for  a  project. 

d.  Investigation  into  what  backgrounds  leads  to  higher  performance 
(production) . 

e.  How  do  you  staff  for  a  changing  project? 


I 


I 
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Direction 

D1  The  direction  function  should  (be/have) : 

a.  Applied  properly  to  effect  a  successful  delivery  of  the  soft¬ 
ware  system. 

b.  (More)  formal. 

c.  Documented. 

D3  The  direction/motivation  function  should  (be/have): 

a.  Motivate  programmers  to  be  aware  of  their  importance. 

b.  Motivate  programmers  to  be  aware  of  being  a  professional. 

D5  Management  should  promote  good  and/or  improved  communications 
(interface)  between: 

a.  Customer  (user)  and  developer. 

b.  System  designers  and  analysts/programmers. 

c.  Management  and  technical  personnel. 

d.  Programming  and  test  personnel. 

D9  Miscellaneous  lessons  learned  and  comments: 

a.  Highly  motivated  inexperienced  staff  can  constantly  out 
perform  less  motivated  senior  staff. 

b.  Never  believe  anything  a  computer  vendor  says. 

DR  A  RiiD  in  direction  should  include: 

a.  How  to  improve  and  maintain  communications. 

b.  Survey  present  methods  and  use  the  best. 
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Control 

Cl  The  control  function  should  (be/have) : 

a.  Provide  exacting  control  and  feedback  over  the  project. 

b.  (More)  formal  function. 

c.  Enforce  the  use  of  controls  already  established. 

d.  Provide  (better)  project  status  visibility. 

e.  An  automated  means  of  monitoring  software  development. 

f.  Modularize  work  into  definable  tasks  (WBS) . 

g.  Control  the  requirement  specification  function. 

h.  Control  project  resources. 

i.  Software  separate  from  hardware. 

j.  Include  race  charting  to  monitor  progress. 

k.  Record  time  vs  activity. 

CV  The  review  process  should  (be/have) : 

a.  Hold  effective  reviews. 

b.  (More)  periodic/excensive. 

c.  Done  at  critical  milestones  in  project. 

d.  Qualified  staff  is  available  for  reviews. 

e.  Review  requirement  specification  for  accuracy. 

CW  Walk-through  should  (be/have) : 

a.  Used  as  a  peer  review  of  the  software. 

b.  (More)  formal. 

c.  Used  more  extensively  (scheduled) . 

CS  Software  standards  should  (be/have): 

a.  Developed  for  more  functions. 

b.  Expanded  to  a  greater  detail. 

c.  Enforced. 

d.  Simplified. 

e.  Monitored. 

f.  Developed  for  software  design. 

g.  Developed  for  programming  (coding). 

h.  Developed  for  documentation. 

i.  Developed  for  debugging  tools. 

j.  Not  have  a  fixed  number  of  lines  of  code  per  product. 
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CM  Sofcware  configuration  managemenc  should  (be/have): 

a.  Provide  control  of  software  undergoing  development. 

b.  (More)  formally  applied. 

c.  Flexable  to  accommodate  required  change. 

d.  Used  (earlier). 

e.  Enforce  strict  adherence  to  baseline. 

f.  Formal  document  changes. 

g.  Control  changes  to  baseline. 

h.  Restrict  software  change  in  requirement  specif icat ions  to 
minor  changes  at  the  start  of  development. 

CT  Test  and  verfication  functions  should  (be/have) : 

a.  .More  formally  applied. 

b.  Emphasized. 

c.  Used. 

d.  Software  error  checking. 

e.  Incorporate  traceability  from  requirement  specification  to 
final  code. 

f.  Applied  top-down. 

CR  R&D  required  in  control  should  include: 

a.  The  development  of  traceability  techniques. 

b.  Research  into  monitoring  a  project. 

c.  Research  into  how  to  test  delivered  software. 

d.  .Methods  to  automatic  configuration  management  aids. 

e.  Research  into  (automatic)  control  techniques,  tools  and 
procedures. 

f.  Automatic  status  reporting  without  frequent  inputs  from 
project  individuals. 

g.  How  to  measure  software  productivity. 


COMPUTERWORLD 

THE  NEWS  WEEKLY  FOR  THE  COMPUTER  COMMUNITY  (Used  With  permission  Of  International 

214  THIRD  AVENUE,  WALTHAM,  MASSACHUSETTS  02154  Data  Corporation,  Waltham,  MA) 

RESEARCH  DEPARTMENT  TEL.  617/890-3770 

June,  1978 

Dear  Census  Respondee, 

The  Compuceruorld  Research  staff  would  like  to  thank  you  for  your 
cooperation  In  responding  to  our  Worldwide  Computer  Installation 
Census.  It  Is  only  through  such  continued  cooperation  that  we  are 
able  to  successfully  monitor  the  growth  trends  of  the  computer 
industry . 

As  an  expression  of  our  appreciation,  we  ace  pleased  to  present 
the  results  of  a  census  report  prepared  by  the  staff  at  our 
publishing  affiliate,  EDP  INDUSTRY  REPORT.  This  "real  world" 
census  Is  the  product  of  a  direct  extrapolation  upon  our  Worldwide 
Computer  Installation  data  base. 

We  hope  you  will  find  the  enclosed  information  extremely  Interesting. 

We  look  forward  to  hearing  from  you  again  next  year. 

Sincerely , 

Computerworld  Research  Staff 

SMALL  BUSINESS  COMPUTER  CENSUS 

Tfi€  of  cortotn  comoutors  is  ■*aifiico«rputtrs"  is  based  on  fnarttetplace  definitions  as  perctived  by  IDC. 

Minis  are  general -ou«*pose  in  desl^  but  sold  as  tools,  not  just  solutions,  are  available  from  the  makfs  as  complete 
systems,  not  just  boards;  are  available  to  OEMs  and  usually  discounted  In  volume  buys;  and  are  oart  Oi'  a  family  that 
las  at  least  one  product  in  the  $2.000-$E5,000  price  range  and  comes  with  at  least  sk  ram.  site  classes  are:  (S) 

Sueemilni;  (T)  Traditional  Mini;  (M)  Mlcromlnl. 
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GENERAL<PURPOSE  COMPUTER  CENSUS 

Counts  'n  me  General -Purpose  Compucer  Census  are  oased  on  me  oest  information  avaiUole  to  IOC  ••  Out  are  estimates 
nevertheless.  Conouter  makers,  as  a  rule,  do  not  directly  supply  this  type  of  data,  'jeneral -purpose  computers  —  as 
cnaracterued  by  ISM's  370.  303X  and  competitors  comprise  the  bulk  of  didital  computers  oy  value.  They  are  pnmar- 
My  character  or  byte  oriented  and  progranned  m  .ni9ner-1eve)  language. 

Instead  of  being  pegged  to  constantly  snifting  average  values,  site  classes  are  based  on  currently  marketed  IBM 
products  and  otner  manufacturers  models  that  compete  with  them.  e.g^. ,  a  computer  in  Size  Class  7  would  compete  with 
an  IBM  3033  or  168.  (Other  classes  can  oe  cnecked  under  IBM's  coTumn.)  Note  that  many  mainframe  manufacturers  also 
offer  smaller  systems  that  are  included  in  the  Small  Business  Computer  Census.  An  'x'  'n  the  last  column  means  none 
on  order  ^rom  the  original  manufacturer. 
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APPENDIX  D 

NARRATIVE  AND  CANDID  (CLEAR  TEXT)  ANSWERS  TO  SELECTED  QUESTIONS 


INTRODUCTION 

This  section  deals  with  actual,  unaltered,  answers  provided  to  a 
series  of  narrative  response  questions,  specifically,  Questions  33,  51, 
52,  67,  68,  90,  115,  127  and  159  of  Part  Two  and  Question  25  of  Part 
Three  (which  has  been  renumbered  164  of  Part  Two)  and  answers  that  were 
provided  to  questions  which  were  not  multiple  choice,  specifically  1,  8, 
9,  12,  28,  29.  69f,  83,  84,  93,  94,  96,  105h,  109,  118,  120,  133,  137, 
and  148.  Also,  gratuitously  included,  at  the  end  of  this  section  are 
relevant  excerpts  from  correspondence  the  authors  had  with  respondents 
and  prospective  respondents  during  the  course  of  survey  development  and 
circulation. 

The  narrative  response  questions  asked  for  opinions  on  how  (or  if) 
the  surveyee  would  have  done  things  differently  had  he  to  do  them  over 
again  (lessons  learned) ,  or  what  form  of  research  he  would  wish  to  see 
undertaken  with  an  eye  toward  improving  the  lot  of  the  project  manager. 
Other  non-multiple  choice  questions  asked  for  answers  for  which  the 
authors  could  not  provide  a  possible  set  of  answers  because  of  the  wide 
variety  of  possible  answers. 

Each  discernible  response,  whether  included  in  the  following  page's  or 
not,  has  also  been  analyzed  and  coded  to  facilitate  entry  into  the 
computer  data  base.  Since  this  reduction  of  comment  to  code  destroyed 
some  of  the  richness  of  prose,  the  authors  felt  it  worthwhile  to  include 
this  "verbal"  section  in  the  report. 

The  answers  as  they  appear  in  the  following  pages  have  been  "cleaned 
up"  to  assure  anonymity  from  the  standpoint  of  author,  firm,  and  project. 
Identical  or  nearly  identical  responses  have  been  eliminated  as  have  in¬ 
complete  (incompleteable)  sentences  and  one  worders.  With  the  exception 
of  the  "clean  up"  and  correction  of  the  most  obvious  spelling  and 
punctuation  errors,  those  responses  included  in  the  following  pages  are 
as  received.  Though  they  do  not  in  every  instance  answer  the  question 
asked,  they  do  relate  to  the  subject.  As  an  aside,  we  make  no  claim  to 
total  understanding  of  every  response. 
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QUESTION  1  What  was/is  your  position  in  relation  to  the  project  you 
are  reporting  on? 

ANSWERS 

I 

The  following  answers  are  the  various  titles  of  personnel  who 
answered  the  que'stionnaire  (grouped  as  to  their  relationship  to  the 
project) . 

a.  Project  Manager 
Project  manager 
Project  leader 
Project  engineer 
Section  manager 
Task  manager 

Responsible  for  computers  system  hardware  and  software 

b.  Software  Project  Manager 
Data  processing  manager 
Manager  of  software  engineering 
Manager  of  software  development 
Software  development  group  leader 
Central  computer  subsystem  manager 
Software  manager 

Responsible  for  the  computer  activities 

Programming  supervisor/mechanization  lead  engineer 

Technical  director  and  director  computer  programming 

Group  leader 

Software  design  manager 

Manager,  data  processing  and  software 

Group  leader  -  software  development 

Group  leader  -  systems  design 

Lead  software  supervisor  ^ 

Assistant,  standard  software  development 
Project  manager  for  software 
Associate  program  manager 
Group  supervisor 

c.  Project  Manager's  Supervisor 
Supervisor  of  project  manager 


Project  Individual 
Analyst 

Software  engineer 

Special  assistant  to  the  project  officer 
Technical  advisor  for  software  acquisition 
Technical  supervisor 


Corporate  Officer/Staff 
Director 

Director,  quality  assurance 
Other  Independent  technical  advisor 
Project  historian 
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QUESTIONS  8,  9,  and  12  If  the  target  (production)  and/or  host  (develop¬ 
ment)  computer  for  this  software  capability  was  an  off-the-shelf 
commercial  system  give;  a.  manufacturer,  make  and  model,  and  b. 
operating  system  employed. 

ANSWERS 


The  first 

column  gives  the 

question  number  answered 

Question 

Manufacturer 

Make  and  Model 

Operating  System 

9 

IBM 

370-125 

DOS-VS 

12 

IBM 

370-125 

DOS-VS 

8 

LEG 

System  III 

LEG  MPOS 

9 

LEG 

System  III 

LEG  MPOS 

9 

ROLM 

1603 

None 

12 

ROLM 

1603 

None 

8 

IBM 

370/158,  168 

VS 

9 

UNIVAG 

1110 

EXEC  8 

8 

UNIVAG 

9480 

None 

8 

GDG 

System  17 

9 

CDG 

System  17 

8 

Raytheon 

704 

9 

Raytheon 

704 

8 

Varian 

V76 

VORTEX  II 

9 

Va^rian 

V76 

VORTEX  II 

12 

Varian 

V76 

VORTEX  II 

9 

NANO DATA 

QM4 

As  provided 

DEG 

PDP-10 

12 

nanodata 

QM4 

As  provided 

DEG 

PDP-10 

8 

GDG 

7600/7700 

SCOPE 

9 

GDG 

7600/7700 

SCOPE 

8 

Honeywell 

316 

9 

GDG 

6600 

SCOPE  4.0 

12 

GDG 

6600 

SCOPE  4.0 

8 

DEG 

11/20,  11/40 

In-house 

9 

DEG 

11/20,  11/40 

In-house 

8 

IBM 

370 

9 

IBM 

370 

8 

IBM 

DOS 

9 

IBM 

DOS 

8 

GDC 

3800 

SYMON  - 

8 

SEL 

8500/8600 

RTM  (modified) 

9 

SEL 

8500/8600 

RTM  (modified) 

Manufacturer 

Make  and  Model 

Operating  System 

CDC 

7700 

SCOPE 

CDC 

SCOPE 

IBM 

360/67 

TSS 

IBM 

360/67 

TSS 

CDC 

175 

NOS 

IBM- 

370-158 

CDC 

6600 

IBM 

360/65 

DATACRAFT 

6024/4 

DATACRAFT 

DATACRAfT 

6024/4 

DATACRAFT 

DATACRAFT 

6024/4 

DATACRAFT 

see 

660 

see 

660 

see 

660 

XEROX 

SIGMA- 5 

XDS 

INTERLATA 

70 

RBM 

IBM 

AP-101 

Special  purpose 

IBM 

360/75 

OS/360 

IBM 

Harris 

Slash  5 

IBM 

360/370 

Harris 

Slash  5 

IBM 

360/370 

Harris 

Slash  5 

IBM 

360/370 

CDC 

CYBER  72 

CDC 

CYBER  72 

IBM 

370-75 

EOS 

IBM 

370-75 

EOS 

UNIVAC 

1108 

UNIVAC 

1108 

XEROX 

SIGMA  5 

RBM 

XEROX 

SIGMA  5 

RBM 

Hewett-Packard 

2100 

RTE 

Hewett-Packard 

2100 

RTE 

IBM 

370/168 

OS  21.6 

INTERDATA 

8/32 

OS  32  MT 

INTERDATA 

8/32 

OS  32  MT 
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Question 

Manufacturer 

Make  and  Model 

9 

IBM 

370-158 

8 

UN I VAC 

1108 

9 

univac 

1108 

8 

IBM 

360/75 

9 

IBM 

360/75 

8 

UNIVAC 

1108 

9 

UNIVAC 

1108 

8 

IBM 

370/168 

9 

IBM 

370/168 

8 

CDC 

6500 

9 

CDC 

6400 

12 

CDC 

6500/6400 

8 

CDC 

CYBER  73 

9 

CDC 

CYBER  73 

8 

CDC 

3800 

9 

CDC 

3800 

8 

IBM 

360/65 

9 

IBM 

360/65 

8 

DEC 

PDP-11/45 

9 

HoneyweLL 

6000 

8 

UNIVAC 

90/60 

9 

IBM 

370-148 

Operating  System 


EXEC  8 
EXEC  8 

OS  (modified) 
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QUESTIONS  28  and  29  In  Question  28,  it  was  asked  -  Was  it  necessary  to 
rewrite  the  specifications  before  proceeding  with  design?  And,  if  the 
answer  was  "yes",  to  specify  the  percent  rewritten.  Question  29  asked  - 
If  it  was  necessary  to  rewrite  the  specifications,  what  was  the  reason? 

.ANSWERS 

The  answer  to  Question  28  is  placed  in  parenthesis  before  the  answer 
to  Question  29.  The  number  is  the  percent  of  requirement  specifications 
rewritten. 

(20%)  Clarification  and  resolve  conflicts  between  specifications. 

(Yes,  it  was  necessary,  but  never  done,  design  documents  were  changed.) 
Change  of  scope,  inconsistency. 

(Specifications  were  written  along  with  design.) 

(20%)  Added  scope. 

(30%)  Discussions  with  customer  during  their  review  of,  or  after, 
their  approval  of  requirement  specifications,  resulted  in  major  require¬ 
ment  changes,  a.g.,  positive  attendance  reporting  by  optical  sensing 
changed  to  excess  reporting  by  batch. 

(60%)  Changes  in  hardware  design  and  errors  in  specifications  (80/20). 

(60%)  Clarifications  and  refinement  of  requirements. 

(80%)  Original  requirements  ambiguous  and  incomplete. 

(50%)  Changes  in  scope. 

(Was  reworked  during  design)  Changes  in  the  objectives  and  sequence 
to  be  employed  in  the  system  demonstration. 

(95%)  Normal  and  expected  interaction  of  total  system  design. 

(50%)  Customers  and  internal  project  dialogue. 

(20%',  significant  change  traffic  continued  after  design  base-line 
frozen,  all  the  way  into  qualification  test.  Resulting  in  about  20% 
change  from  original  base-line  design.) 

(100%)  Specifications  absolutely  unuseable  -  ambiguous,  not  test¬ 
able. 

(50%)  Continuous  Government  redirection  and  program  rescoping. 

(50%)  Customer  requirements  changed  or  expanded. 

(15%)  Refinement  of  performance  of  objectives. 

(20%)  Changing  requirements  and  inconsistent  specifications. 

(50%)  Lack  of  details;  omitted  many  essential  characteristics, 
especially  interfaces. 

(80%)  To  insure  that  the  functional  requirements  were  understood  to 
permit  further  design. 

(15%)  Changing  requirements. 
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(10%)  Increased  system  knowledge. 

(Small  number  of  changes,  but  customer  had  some  add-systems.) 

(25%)  Customer  direction,  mistakes,  too  much  use  of  preliminary 
requirements. 

(20%)  System  not  fully  understood. 

(30%)  Maturity  of  the  requirements  (block  update) . 

(30%)  To  make  the  job  [more]  easily  done  or  to  overcome  some 
difficulties  which  were  identified  later. 

(90%)  Incorporate  specifics  in  place  of  functional  requirements. 

(25%)  Changes  directed  by  customer. 

(50%)  [changed  from]  Manual  to  computerized  procedure  of  testing. 

(100%)  Additional  technical  insight. 

(20%)  Consistency  and  change  in  tactics. 

(40%)  System  would  not  work. 

(40%)  Changes  in  scope,  errors,  lack  of  customer  expertise. 

(20%)  Clarification  and  improvement. 

(75%)  Lack  of  sufficient  detail. 

(20%,  2  phase  operation  -  existing  operating  system  installed  in 
first  phase  -  no  change  -  new  operating  system  with  new  bells  and 
whistles  required,  requiring  approximately  20%  rewrite  due  to  learning 
curve  response.)  Customer  became  familiar  with  capability  and  custom¬ 
ized  to  fit  his  particular  need. 

(100%)  Only  very  broad  requirements  were  furnished  by  user. 

(100%,  New  specifications  were  written  after  the  first  design  failed!) 
Total  system  was  inoperative  due  to  system  (both  hardware  and  software) 
design  flaws. 


(QUESTION  33  If  you  could  affect  the  method  by  which  requirements  are 
specified,  or  were  able  to  initiate  research  into  improving  the 
requirement  specifications  function,  what  action  would  you  take? 

ANSWERS 

Define  a  formal  "requirements  phase"  whose  output  would  be  a 
contractor/customer  agreed  upon  baseline  for  design  work. 

I  would  add  dynamic  simulation  capabilities  to  a  package  like 
CARA-URL/URA  or  PSL/PSA. 

Make  sure  it  is  a  joint  commitment  and  effort  by  customer  and 
developers . 

The  method  is  relatively  unimportant,  unless  the  customer  wants  to 
impose  a  design.  Otherwise,  any  readable  statement  of  the  requirements 
will  do,  as  long  as  it  is  clear  and  complete. 

Insure  accessibility  of  key  people  to  review  specifications  as  to 
accuracy . 

Establish  incremental  requirements  development  compatible  with  total 
project  development. 

Joint  contractor/customer  preparation  of  requirement  specifications 
of  all  ICDs  for  agreeing  to  cost  or  schedule. 

A  more  complete  and  relatively  unbiased  study  should  be  made  of 
various  proposed  techniques  being  advocated. 

If  possible,  the  system  definition  should  be  firmed  up  early  so  that 
the  changes  at  the  start  of  software  development  are  only  minor  adjust¬ 
ments  rather  than  redirection  of  the  effort. 

MIL-STD-483  is  dismal  in  that  the  Part  l(b5)  specifications  outline 
calls  for  too  much  detail  and  actually  obscures  requirements.  For 
example;  An  equation  is  not  a  requirement  but  merely  a  shorthand  state¬ 
ment  of  a  relationship.  Most  people  don’t  know  how  to  write  a  require¬ 
ment  specification.  The  usual  result  is  a  description  of  the  process. 

Intense  interface  with  user  community.  In  this  case,  the  system  was 
procured  by  one  branch  for  use  by  another.  The  second  branch  was  not 
brought  into  the  picture  until  the  acceptance  phase  -  disaster . 

Institute  formal  requirements  methodology,  testable  and  analyzable. 

Promote  more  implementer  involvement  in  specifications  writing  and 
more  structured  specifications. 

More  vigorous  application  of  structured  design. 

Expand  the  team  concept  (vendor  and  customer)  so  that  complete 
understanding  may  be  fostered  on  both  sides. 

Coding  should  not  proceed  until  requirements  are  complete.  Require¬ 
ments  should  be  more  complete  and  should  be  written  (in  part)  by  end 
users  (mission  operations  team)  in  conjunction  with  customer /engineer/ 
programmers. 
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Develop  traceability  techniques. 

Require  complete  set  of  specifications  before  initiating  software 
design  and  implementation.  Use  HOL  source  code  as  primary  design 
document . 

With  respect  to  mission  software,  requirements  should  be  developed 
jointly  with  the  customer  prior  to  contract  award.  This  is  perhaps  the 
best  possible  way  to  go. 

Improve  communications  between  systems  design  groups  and  software 
project. 

Requirements  should  be  known  early  enough  to  allow  time  for  review. 

Requirement  specifications  should  be  prepared  by  users/operators/ 
development  team.  Keep  teams  small. 

Invoke  operation  oriented  description  of  performance  requirements. 
Keep  design  detail  out  of  requirement  specifications. 

More  control  to  establish  an  earlier  baseline. 

Requirement  specifications  should  be  developed  top-down  with  trace- 
ability  at  all  levels. 

Elimination  of  redundancy  in  documentations  by  making  B-5  specific¬ 
ations  first  version  of  C-5  type  specifications  (Refer  to  Mi.--Standard 
490).  Use  Parnas'  Hiding  Assumptions  list  to  define  basis  for  choice 
of  modules  and  for  good  user  communications. 

Involve  software  engineers  early  in  the  system  definition  and  allow 
for  more  "up-front"  simulation. 

Insure  that  the  user  specifies  the  functions  that  the  system  must 
perform  and  that  the  developer  obtains  a  full  understanding  of  these 
required  functions. 

Improve  organization  of  requirements  hierarchy  to  facilitate  top- 
down  implementation. 

I  would  revise  MIL-STD-483  to  limit  Part  1  CPCI  specifications  to 
performance  requirements  and  associated  contractor/GFE  interfaces,  and 
include  detail  algorithm  definition  -  equations  (coding  requirements) 
as  a  part  of  the  Part  2  CPCI  specifications. 

More  detailed  system  design  specifications  prior  to  start  of  software 
design. 

Earlier  system  definition,  and  a  software  design  freeze  at  some 
point  during  the  development. 

Would  remove  all  intermediate  agencies  and  monitoring  functions 
from  between  the  customer  (user)  and  the  analytical  functional  designer/ 
programmer. 

Requirements  should  be  verified  as  to  "testability"  prior  to  release. 
Also,  contractors  should  be  allowed  to  participate  during  source 
selection  in  the  requirement  specifications  process. 
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Top-down  documentation,  with  baselines,  formal  change  procedures, 
and  more  involvement. 

Extend  the  period  of  requirements  definition;  automate  the  analysis 
of  requirements  statements  for  completeness  and  consistency. 

A  detailed  requirement  specifications  should  have  included  a  great 
deal  of  analysis,  trade-offs,  simulation  (perhaps)  before  being  started. 
A  formal  SRR  should  then  take  place.  CM  should  be  employed  for  all 
changes  to  requirements  (on  going),  thereafter,  or  in  other  words:  1. 
Emphasize  completeness/thoroughness  of  first  detailed  requirement 
specifications,  2.  Be  fle^tible  for  constant  requirement  changes  there¬ 
after  -  which  may  impact  schedule;  however,  CM/CCB  will  decide. 

Insist  on  complete  identification  of  requirements  and  establishment 
of  configuration  control  over  them  prior  to  proceeding  to  the  next 
phase  of  development. 


QUESTION  51  If  you  had  tocal  control  of  the  planning  function  within 
your  organization,  or  were  able  to  initiate  research  into  how  to 
improve  the  planning  function,  what  actions  would  you  take? 

ANSWERS 

Increase  the  planning  phase  -  complete  detail  design,  prior  to  code 
initiation.  Require  full  system  walk-throughs. 

Initiate  more  detailed  level  of  planning.  Improve  cost  and  schedule 
estimates. 

Based  upon  empirical  data,  I  would  initiate  research  which  would 
result  in  the  development  of  highly  accurate  cost  estimating  methodology 
for  all  phases  of  software  development. 

Careful  planning,  correct  design,  periodic  review  and  testing. 
Verification  and  documentation  should  be  necessary  actions  to  be  taken. 

Increase  system  designers  awareness  of  software  during  system  design 
(trade-off) . 

More  concentration  on  staffing  and  skill  mixes. 

Place  more  emphasis  on  the  requirements  of  supporting  resources 
(computer  utilization,  size,  speed,  etc.).  Scope  the  development 
facility  as  to  its  adequacy  to  support  Che  application. 

Stop  customer  from  freezing  the  design  before  detail  coding 
(programming)  requirements  are  complete. 

Better  staff  training  -  better  communications  with  systems  engineering. 

Combine  all  ai.i’lysts/programming  activities  under  one  control  group. 

I  would  organize  a  highly  proficient  planning  team  from  management 
and  technical  personnel  to  develop  project  plans. 

Place  a  Sw.i.vor  software  specialist  on  staff  in  top  management  to 
assist  in  decision  making. 

Start  planning  earlier. 

Assure  that  system  definition  and  implementation  allow  adequate 
design  and  development  time. 

Devote  more  effort  to  planning.  Involve  programmers /analysts  to  a 
greater  degree.  Require  full  documentation  of  plan.  Continue  to 
review  plan  as  development  proceeds. 

Various  activities  must  be  prioritized  before  detailed  action  is 
undertaken. 

Emphasize  involvement  and  contribution  of  the  "people  doing  the  work" 
in  the  planning  function. 

Allow  more  time  for  planning. 

Design  tools  for  building  emulators. 

Analyze  applicability  of  the  University  of  Michigan  System  Design 
Language . 
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Survey  available  methods;  apply  most  promising  and  weed  out  losers. 

Allocate  greater  percentage  of  the  time  to  planning. 

Set  up  data  base  from  all  ongoing  programs  which  would  be  available 
to  new  programs. 

Reduce  the  number  of  people  and  increase  the  calendar  time. 

Set  means  to  provide  total  access  and  communication  with  existing 
in-company  experience. 

Expand  the  coverage  and  contractual  significance  of  the  computer 
program  development  plan  CPDP  as  defined  in  APR  800-14.  Insist  that  a 
CPDP  be  outlined  during  the  competitive  phase  to  assure  a  credible  bid 
(fixed  price  plus  incentive) . 

Require  top  level  software  design  and  module  interface  specific¬ 
ations  prior  to  detail  software  design. 

Early  system  definition  and  software  design  freeze. 

Capture  data  from  ongoing  programs.  Transfer  knowledge/ techniques 
among  projects,  i.e.,  standard  planning,  measurements,  reporting  and 
estimating  techniques. 

Complete  detailed  development  plans  and  procedures  early  in  the 
project. 

Demand  that  validated  project  management  procedures  be  employed. 

More  senior  management  personnel  should  be  involved. 

Collect  software  metrics  on  productivity  and  quality. 

Increase  user  involvement  in  designing  and  testing  phases. 
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QUESTION  52  If  it  were  in  your  power  to  make  changes  in  the  way 
technical  decisions  are  made  concerning  programming  techniques,  test 
procedures,  documentation  standards,  etc.,  what  action  would  you  take? 

ANSWERS 

More  emphasis  on  modern  structured  technique  and  automated  tools. 

Get  dollars,  survey  available  methods,  apply  most  promising  and 
weed  out  losers. 

Study  and  evaluate  variety  of  modern,  automated  techniques  to  be 
ready  fo*'  appropriate  application. 

I  woul_  revent  outside  "experts"  from  imposing  their  pet  techniques 
and  controls  in  areas  where  they  are  inappropriate,  misguided,  and 
costly. 

Use  more  junior  programmers  for  trivial  aspects  of  the  job. 

More  emphasis  on  phase  design  effort  and  better  planning  for  testing. 
Development  of  more  test  aids  such  as  program  flow  and  structure 
analyzer  and  aids  to  the  programmer  for  fixed  point  machines  as  long  as 
we  continue  to  use  fixed  point  for  flight  equipment. 

Review  what's  in  general  use  and  publish  recommended  methods  as  a 
standard.  No  such  document  currently  exists. 

More  detailed  plans  must  be  made  and  reviewed  before  actual  coding 
begins.  Short  range  goals  must  not  be  allowed  to  dominate  planning 
function. 

Use  standard  industry  techniques  for  flight  software  development. 

More  automated  tools,  more  formal  walk-throughs. 

Pass  all  "technologies"  through  technical  control  panel  for  assess¬ 
ment  prior  to  use  on  project. 

Greater  standardization  throughout  the  project.  More  rigid 
monitoring  and  enforcement. 

Have  more  desk  checking. 

Develop  design  validation  techniques. 

Select  one  able,  right  person  (who  has  experience  in  programming 
techniques,  as  well  as  management)  to  be  program  manager.  Give 
direction  and  guidance  concerning  objectives,  procedures,  schedules, 
etc,  (other  necessary  actions) .  The  program  manager  will  take  care  of 
the  rest. 

Increase  programmers'  awareness  of  their  importance,  and  the 
importance  of  their  professional  demeanor. 

I  would  improve  the  entire  spectrum  of  project  standards  to  be 
enforced. 

Structured  programming  concepts  and  chief  programmer. 

More  top-down  structured  design. 
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Insist  on  a  common  language  where  practical.  Include  test  require¬ 
ments  as  an  integral  part  of  the  requirement  specifications. 

Simplify  the  number  of  different  standards  in  use. 

I  would  give  additional  impetus  to  all  aspects  of  testing,  giving  it 
equal  status  with  other  major  phases  of  software  development,  allocating 
perhaps  40%  of  project  budget  to  this  activity.  I  would  insist  upon 
certain  standards  associated  with  documentation  content,  but  relax 
format  considerations. 

Increased  formalization  of  techniques.  Too  much  "by  the  seat  of  the 
pants"  planning,  dicision  making  takes  place.  Much  of  this  is  due  to 
"the  first  time"  problem  -  remaining  due  to  an  unstructured  environment. 

Select  a  simple  set  of  rules  for  program  organization  and  structure 
and  concentrate  on  making  programmers  follow  them. 

.Assure  that  past  experience  and  techniques  are  considered.  Document 
rationale  .for  all  decisions. 

Set  means  to  provide  total  access  and  communication  with  in-company 
experience. 

Eliminate  requirement  for  detailed  flow  charts  prior  to  coding. 

Use  seif  annotating  listings  in  lieu  of  flow  charts. 

More  reviews,  walk-throughs  and  configuration  management  procedures. 

Follow  more  structured  software  development  approaches.  We  are 
planning  to  do  this  for  our  FDC  delivery. 
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QUESTION  67  Which  of  Che  many  forms  of  project  organization  do  you 
feel  contributes  the  most  to  the  success  of  the  project? 

ANSWERS 

Best  organization  is  dictated  by  the  job  to  do,  [and]  the  talents  of 
specific  people  available. 

Matrix. 

Task  oriented. 

Single  clear  leadership  role. 

The  lead  programmer,  single  architect  approach. 

Direct  control,  multi-function,  integrated  program/ analyst/manager 
teams. 

Matrix,  with  strong  technical  leadership. 

Each  form  has  appropriate  application  under  different  conditions, 
constraints,  and  objectives. 

Management  by  personnel  who  understand  both  technical  and  adminis¬ 
trative  areas. 

Program  manager  and  a  technical  manager. 

Project  organization.  , 

Line  organization. 

A  changing  organization  that  adapts  to  the  work  being  performed. 

Line  organization  responsible  for  all  programming  functions. 

Matrix  organization  for  overall  ADP  organization  -  structuring  and 
dedicating  project  teams  to  individual  projects. 

Chief  programmer  -  acts  as  interface  from  line  organization  into  the 
rest  of  the  matrix. 

Team  organization. 

Line  organization  with  three  independent  sub-groups;  requirements/ 
analysis,  programming/software  system  design,  and  test. 

Mixing  of  disciplines  at  the  lowest  level. 

Systems  -  subsystems  by  functional  areas. 

Matrix. 

One  in  which  the  project  manager  has  both  programming  and  technical 
responsibilities  with  permanent  assignment  of  personnel. 

A  project  manager  with  full  authority  for  the  project.  Use  of  a 
chief  programmer  subordinate  to  the  project  manager. 


QUESTION  68  If  you  had  ic  within  your  power  to  make  one  change  in  the 
way  the  project  was  organized,  what  action  would  you  take,  or  if  you 
had  resources  available  to  undertake  research  in  any  area  of  project 
organization  which  aspect  would  you  explore? 

ANSWERS 

Better  planning. 

More  planning. 

Visibility  of  project  status. 

Doesn't  really  require  any  research.  .Matrix  or  functional  will 
work  equally  well  depending  on  the  size  of  the  project. 

Implement  the  team  approach  .-ith  the  programmers,  back-up  programmers, 
and  librarians  as  identifiable  entities. 

The  extent  to  which  technical  and  administrative  functions  should  be 
separated  should  be  explored. 

Permanently  assigned  staff  for  duration  of  project. 

More  technical  staff  direction. 

Implement  a  separate  test  function  at  project  initiation.  Create  a 
separate  documentation  function. 

Would  explore  planning  (scheduling)  and  testing  organization. 

Provide  sufficient  calendar  time  for  each  group  to  complete  its  task 
without  -vertime.  Particular  emphasis  must  go  on  requirements  and 
specifications. 

Metrics;  so  we  talk  less  and  know  more. 

I  would  spend  the  necessary  amount  of  time  thoroughly  researching 
available  empirical  data  to  arrive  at  the  most  accurate  possible 
methodology  for  estimation  of  software  development  cost  (all  phases) . 

Software  activities  report  to  higher  level  within  project. 

Use  chief  programmer  teams. 

Would  place  hardware  and  software  totally  subservient  to  system. 

Include  a  separate  software  Interface  team  (a  group  responsible  for 
the  top  level  software  design) . 

Would  employ  separate  software  test  team  that  is  not  under  control  of 
the  software  manager. 

Program  status/productivity  measurement  aids;  automated  configuration 
management  aids. 

Assign  a  thoroughly  experienced  project  leader  at  the  very  beginning. 
Validation  of  proposed  project  management  methods. 

No  changes  in  organization.  There  were  some  things  the  organization 
should  have  done  differently. 
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Assign  stronger  technical  personnel  to  lead  task,  manager  positions. 
I  wouldn't  have  had  a  personnel  cut  just  prior  to  the  effort. 
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QUESTION  69 f  The  project  manager  was  appointed,  or  selected  for  this 
project  by: 

ANSWERS  Grouped  by  relationship  to  project  manager. 

a.  Senior  ADP  Manager 

Senior  ADP  manager  through  experience 
Assistant  manager,  engineering  operations 
Manager,  software  development  laboratory 
Senior  ADP  manager 
Project  reviewing  authority 

The  project  review  authority  as  delegate  of  division  general 
manager 

Division  general  manager  (V.P.) 

ADP  manager 
Line  manager 
Manager  two  tiers  up 

Support  organization  manager  with  the  approval  and  concurrence 
of  the  program  manager 

Line  manager  for  software 

Senior  line  ADP  manager 

Assistant  director  business  systems  development 
Director  of  operations 
Functional  section  chief 
Department  manager/Division  manager 
Department  chief 

b.  Senior  Non-ADP  Manager 
Director  of  development 
Director  of  engineering 
Avionics  chief  engineer 
Director 

Line  organization 
Department  management 
Department  head 
The  avionics  manager 
Systems  project  engineer 
Project  engineer 
S<-ction  chief 
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c.  Program  Manager  (as  opposed  co  projecc  manager) 
Program  manager 

The  program  manager  of  che  total  project 

d.  Senior  Corporation  Officer 
Vice  President 
President 

Senior  management 
Program  Vice  President 
Upper  management  selection 
Company  general  manager 

The  division  vice  president  for  Federal  Systems 
He  was  appointed  by  top  level  management 
Assistant  vice  president 
The  line  executive 

e.  Customer 
Customer 
SPO  chief 

f .  Other 

Ex  officio 

Agreement  between  line  and  staff  management 
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QUESTIONS  83  and  84  Question  83  asks  what  percent  of  the  progranimer/ 
analyst  staff  were  originally  computer  operators  who  moved  directly 
from  machine  operations  to  programming.  Question  84  asks  if  the  former 
operators  make  successful  programmers. 

ANSWERS  The  answer  to  Question  83  -  the  %  of  operators  who  became 
programmers  -  is  in  parentheses  before  the  answer  to  Question  84. 

(0%)  Very,  if  they  get  the  BA  or  BS  required  for  the  promotion. 

(5%)  The  transition  from  operators  to  programmers  was  very  success¬ 
ful,  especially  in  the  area  of  JCL  procedures. 

(0%)  In  scientific  programming  area  there  has  been  less  chance  for 
machine  operators  to  become  programmers.  I  haven't  seen  any. 

(10%)  Yes,  the  Best!! 

(10%)  Yes,  and  their  familiarity  with  OPS  problems  and  procedures 
was  useful. 

(0%)  We  have  used  this  technique  in  other  areas  of  the  matrix 
organization  -  usually  they  work  best  in  an  operation-system  mainten¬ 
ance  function  -  i.e. ,  recurring  versus  design. 

(1%)  One  did. 

(15%)  Average  in  quality. 

(0%)  Avionics  work  emphasizes  engineering  skills,  not  programming 
skills. 

(UNK)  I  remember  one  who  turned  out  to  be  very  good. 

(4%)  Yes,  good/average. 

(5%)  Yes,  but  limited  in  scope. 

(55%)  2  proficient 

1  mediocre 

2  incapable 

(2%)  Yes,  their  long  experience  as  operators  on  these  systems  and 
their  knowledge  of  policies  help  them. 

(5%)  Yes. 
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QUESTION  90  If  it  were  within  your  power  to  make  any  changes,  or 
initiate  any  research  in  Che  area  of  staffing,  what  would  you  consider 
the  most  fruitful  area  for  modification  or  study? 

ANSWERS 

How  to  maintain/ improve  communications. 

Experiment  with  more  specialized  staffing;  i.e. ,  with  a  number  of 
more  functional  roles  rather  chan  a  group  of  programmers/analysts . 

Investigate  what  personnel  backgrounds  lead,  or  tend  to  lead,  to 
high  performance  ADP  personnel. 

Use  different  categories  of  programmers  with  different  levels  of 
expertise  for  appropriate  aspects  of  the  project. 

Have  a  sufficient  number  of  "open  ended"  research  and  sustaining 
programs  to  serve  as  buffers  for  personnel  so  as  to  provide  more 
flexibility  in  staffing. 

Staff  should  not  build  up  too  fast.  Pure,  high-level  people  for 
initial  design  might  be  best. 

Majority  of  the  problems  encountered  involved  outside  pressures  - 
such  as  removing  personnel  for  short  periods  to  handle  brush  fires  - 
matrix  organization  has  solved  that  problem.  Matrix  organization  has 
also  improved  overall  training  program  which  now  takes  place  in  a 
functional  area  -  eliminating  much  pressure  from  the  individual  projects. 

Study  the- number  of  librarians  required  for  a  job  of  a  particular 
size. 

Need  better  method  of  evaluating  background/experience  vs  project 
requirements . 

Training  in  design  techniques. 

Add  programmer  aid  or  support  librarian  to  the  team. 

More  formal  traiing  in  the  engineering  applications. 

Training  in  new  technologies. 

Identify  the  quality  and  training  requirement  of  persons  to  be 
hired  early  in  project  start-up. 

I  would  like  my  team  members  to  get  well  trained  if  anyone  is  in 
need  of  it. 

Staffing  for  an  evolving  project,  i.e. ,  soft  schedule. 

Personnel  screening  techniques  to  minimize  the  on-the-job-training . 

Replacing  personnel. 

Software  staffing  must  include  a  cross  section  of  personnel  types. 
Hardware /Software  background  vs  software  only,  experience  vs  youthful 
ideas,  etc.  -  Study  to  determine  the  factors  which  most  affect  the 
proper  mix  for  a  given  project. 
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The  balance  of  young  Co  old  and  experience  to  inexperience. 

Participate  in  personnel  selection. 

When  you  are  in  trouble  a  progranuner  is  not  a  programmer  if  he 
doesn't  know  anything  about  the  hardware  and/or  software  to  be  developed. 

Back  to  the  drawing  board. 

Measurement  of  productivity  and  correlation  with  selection  factors. 
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QUESTION  93  Which  manual  reporting  procedures  were  used  in  project 
monitoring  and  management?  At  what  level  did  they  originate,  and  how 
high  did  they  go?  How  often  were  they  aggregated,  condensed,  or  edited 
as  they  moved  up  the  chain? 

ANSWERS 


Report  Title 

Lowest 

Originator 

Wkly  Activity 

Proj  status 
Significant  chg 

Worker 

Sect  mgr 

Proj  mgr 

Wkly  Activity 

Programmer 

Wkly  Activity 

Proj  status 

Sect  head 

Proj  mgr 

Wkly  Activity 

Proj  status 

Proj  leader 
Proj  leader 

Wkly  Activity 

Proj  status 

Anal /Prog 

Mgr 

Wkly  Activity 

Proj  status 
Significant  chg 

Programmer 

Ld  Prog/Anal 
Programmer 

Wkly  Activity 

Proj  status 

Programmer 

Sr  Anal 

Wkly  Activity 

Proj  status 
Significant  chg 

Programmer 
Proj  mgr 

Tech  leader 

Wkly  Activity 

Proj  status 
Significant  chg 

Prog/Anal 

Prog/Anal 

Tech  leader 

Wkly  Activity 

Proj  status 

Programmer 
Proj  mgr 

Wkly  Activity 

Proj  status 

Cost  vs 

Performance  Rpt 

Programmer 
Proj  engr 

Fin  Anal 

Wkly  Activity 

Proj  status 

Programmer 
Software  engr 

Wkly  Activity 

Proj  status 
Significant  chg 

Branch  Chief 
Branch  Chief 
Branch  Chief 

Wkly  Activity 

Proj  status 
Significant  chg 

Gp  leader 

Proj  engr 

Proj  engr 

Highest 

No.  of 

Recipient 

Aggs/ Edits 

Proj  mgr 

2 

Proj  mgr 

1 

Proj  mgr 

- 

Proj  mgr 

3 

Proj  mgr 

1 

President 

2 

DP  Admin  Chief 

- 

DP  Admin  Chief 

- 

Gen  mgr 

2 

Gen  mgr 

1 

Gen  mgr 

3 

Gen  mgr 

3 

Proj  mgr 

3 

Division  VP 

1 

Division  Pres 

3 

Proj  mgr 

1 

Sr  mgmt 

1 

Sr  mgmt 

1 

Proj  mgr 

2 

Proj  mgr 

2 

Proj  mgr 

2 

Proj  mgr 

0 

Division  mgr 

2 

Proj  engr 

0 

Cust/Div  mgr 

2 

Proj  engr 

0 

Software  engr 

0 

Computer  sys  mgr 

Director 

Director 

Director 

0 

VP 

4 

VP 

3 

VP 

2 
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Lowest 

Highest 

No.  of 

Report  Title 

Originator 

Recipient 

Aggs/ Edits 

Wkly  Activity 

Subsys  mgr 

VP 

1-2 

Proj  status 

Subsys  mgr 

Customer 

2 

Significant  chg 

Wkly  Activity 

As  required 

Lead  engr 

Proj  mgr 

4 

Proj  status 

Proj  mgr  SW 

Proj  mgr 

2 

Significant  chg 

Proj  mgr  SW 

Proj  mgr 

2 

Wkly  Activity 

WPM 

Proj  mgr 

1 

Proj  status 

WPM 

Customer 

2 

Wkly  Activity 

Prog/Anal 

Proj  mgr 

Proj  status 

Prog/Anal 

Proj  mgr 

Significant  chg 

Prog/Anal 

Proj  mgr 

Wkly  Activity 

Programmer 

Proj  Officer 

Proj  status 

Programmer 

Proj  Officer 

Significant  chg 

Programmer 

Proj  Officer 

Wkly  Activity 

Proj  mgr 

Engr  VP 

4 

Monthly  status 

Proj  mgr 

Division  Pres 

5 

Proj  status 

Sect  head 

Proj  leader 

Wkly/Mo 

Monthly  CSCS/R 

Funct  engr 

Proj  mgr 

reported 

Wkly  Activity 

Team  chief 

Programmer 

1 

Proj  status 

Team  chief 

VP 

2 

Significant  chg 

Team  chief 

VP 

2 

Wkly  Activity 

Programmer 

Proj  mgr 

0 

Proj  status 

Chief  Prog 

Proj  mgr 

0-1 

Wkly  Activity 

Staff 

Program  mgt 

0 

Proj  status 

Program  mgt 

Customer 

0 

Significant  chg 

Staff 

Proj  mgr 

0 

Proj  status 

Programmer 

Program  mgr 

Significant  chg 

Programmer 

Program  mgr 

Wkly  Activity 

Supvr  (gp  engr) 

Proj  mgr 

2 

Proj  status 

Prog  Cont  org 

VP 

1 

Significant  chg 

Mgr 

Proj  mgr 

2 

Wkly  Activity 

Designer 

Customer 

Proj  status 

Designer 

Customer 

Significant  chg 

Designer 

Sys  prog  mgr/cust 

Wkly  Activity 

Programmer 

Dept  head 

Proj  status 

Prog  cont  officer 

Site  director 

Significant  chg 

Funct  dept  head 

Program  mgt 

Wkly  Activity 

Gp  leader 

Program  mgr 

3 

(wkly  prog  rpt) 
Proj  status 

Programmer 

Gp  leader 

1 
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Lowest 

Highest 

No.  of 

Report  Title 

Originator 

Recipient 

Aggs/Edits 

Wkly  Activity 

Gp  leader 

Prog  director 

5 

Proj  status 

Gp  Leader 

Customer 

6 

(monthly  rpt) 

Wkly  Activity 

Branch  prog 

Dept  head 

1 

Proj  status 

Proj  mgr 

Dept  head 

0 

Significant  chg 

Proj  mgr 

Dept  head 

0 

Wkly  Activity 

Prog/Anal 

Top  mgr 

3 

Proj  status 

Prog/ Anal 

Top  mgr 

3 

Wkly  Activity 

Individual 

President 

6 

Proj  status 

Team  leader 

President 

5 

Wkly  Activity 

Chief  prog 

Proj  mgr 

1/wkly 

Proj  status 

Proj  mgr 

President 

Wkly  Activity 

Gp  head 

Proj  mgr 

0 

Proj  status 

Proj  mgr 

AVP 

1 

Significant  chg 

Proj  mgr 

AVP 

1 

Wkly  Activity 

Gp  head 

Program  mgr 

2 

Proj  status 

Gp  head 

Program  mgr 

2 

Significant  chg 

Individual 

Sect  head 

1 

Wkly  Activity 

Sect  head 

Program  mgr 

1 

Proj  status 

Sect  head 

Program  mgr 

1 

Significant  chg 

Sect  head 

Program  mgr 

1 

Proj  status 

Cog  engr/cog  prog 

Sect  chief 

Significant  chg 

Cog  engr/cog  prog 

Sect  chief 

Proj  status 

Cog  engr 

Proj  mgr 

1 

Wkly  Activity 

Gp  leader 

VP 

1 

Proj  status 

Proj  mgr 

VP 

1 

Significant  chg 

Proj  mgr 

VP 

1 

Wkly  Activity 

Team  as  input 
by  individuals 

Program  mgr 

1 

Proj  status 

Team 

DOD/ADP  VP 

3 

Significant  chg 

Team 

DOD/ADP  VP 

3 

(schedule  variance) 

Wkly /Mo  Activity 

Programmer 

Proj  mgr 

0 

Proj  status 

Prog  mgr 

Division  mgmt 

Wkly  Activity 

Indiv  contrib 

Proj  mgr 

2 

Proj  status 

Work  unit  Idr 

Proj  mgr 

1 

Significant  chg 

Work  unit  Idr 

Proj  mgr 

1 

Proj  status 

Proj  mgr 

Directorate 

1 

Wkly  Activity 

Branch 

Director 

Proj  status 

Division 

President 

Significant  chg 

Division 

President 

1 
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Lowest  Highest 

Report  Title  Originator  Recipient 

Wkly  Activity 
Proj  status 
Significant  chg 


No.  of 
Aggs/Edits 


Task  mgr 
Task  mgr 
Task  mgr 


Dept  mgr 
Division  mgr 
Division  mgr 


1 

1 

1 
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QUESTION  94  Which  automated 
monitoring  and  management? 

reporting  systems  were 

used  in  project 

ANSWERS 

Lowest 

Highest 

System 

Originator 

Recipient 

Manho ur / Ac t iv i t y 

Worker  (time  card) 

Froj  mgr 

Manhour/Activity 

Programmer 

Proj  mgr 

Manhour/Activity 

Lead  programmer 

Proj  mgr 

Manhour /Customer 

Programmer 

Next  level 
above  proj  mgr 

Manhour/ Activity 

Programmer 

Proj  mgr 

Manhour/ Activity 

Prog  analyst 

Proj  mgr 

Manday/Task 

Prog  analyst 

Proj  mgr 

Manhour /Ac  tivity 

Accounting 

Proj  mgr 

Manday/Task 

Accounting 

Proj  mgr 

Manhour /Ac tivi ty 

Fin  analyst 

Proj  engr 

Manday/Task 

Fin  analyst 

Proj  engr 

Manhour /Activity 

Software  engr 

Computer  subsys 
mgr 

Manhour /Ac tivity 

Engineer 

Program  mgr 

Manday/Task 

Engineer 

Branch  chief 

Manhour  by  major  activity 

Manday/Task 

Individual 

Cumulative  to 

VP 

Manhours/WBS  Task 

Computer  report 

Proj  mgr 

Manhours /Task 

Prog/analyst 

Proj  mgr 

Manhour/Activity 

Functional  engr 

Proj  mgr 

Manhour /Activity 

Individual 

Prog  cont  org 

Manhour /Activity 

Programmer 

Proj  mgr 

Manday/Task 

Prog  control  officer 

Prog  mgr 

Manhour  by  job  charge 
(this  is  related  to  broad 
tasks  such  as  programming, 
subsystem  equipment  design, 
etc.) 

Programmer 

Prog  mgr 

Manday/Task 

Group  leader 

Proj  engr 

Manhour/Job  charge 

Programmer 

Programmer 

Manhour/Job  charge 

Programmer 

Programmer 

Manhour/Job  charge 

Programmer 

Prog  mgr 
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System 

Lowest 

Orisinator 

Highest 

Recipient 

Manhour /Activity 

Sr  programmer 

Dept  head 

Manday/Task 

Prog  mgr 

Manhour /Activity 
Manday/Task 

Time  card ,  WBS 

Team  members 

Proj  mgr 

ADP  VP 

Cost  data 

Time  cards  by 
individual 

Division  mgr 

Manhour/ Activity 

Individual 

Staff 

Person-Hour  by  Function 

Integration 

Proj  mgr 
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QUESTION  96  List  productivity  indexes  such  as  lines  of  code,  program 
errors,  sources  of  errors,  turn  arounds  required  per  completed  task, 
etc. ,  that  were  employed  in  monitoring  performance. 

ANSWERS 

Documented  pages,  threads,  computer  time  used,  problem  reports/ 
corrections,  lines  of  code. 

Lines  of  code  -  tracked;  not  absolute  measures. 

8  lines  of  code  per  day. 

Productivity  monitored  at  module  level,  since  lines  of  code  per 
module  were  constrained  by  standards.  Complexity  factors  weighed  in. 
Performance  then  evaluated  on  turn-around,  correctness,  efficiency. 

Lines  of  code,  number  of  compilable  units. 

Lines  of  source  code,  number  of  modules. 

Lines  of  completed  code/ per  manhour. 

Lines  of  code,  error  reports,  machine  time  utilization. 

Program  errors. 

Lines  of  code,  program  errors,  sources  of  errors,  turn-arounds 
required  per  completed  task. 

Modules  completed,  lines  of  code. 

We  indeed  use  such  indexes.  We  are  currently  establishing  a  data 
base  of  such  indexes. 

Lines  of  checked  out  and  documented  machine  executable  code  per  hour. 
Lines  of  code. 

Rate  charting  for  module  productivity. 

Program  errors  and  lines  of  code. 

Dollars/lines  of  operational,  documented  software. 

Unfortunately  -  none. 

Did  not  use  productivity  i.idexes. 

Lines  of  code. 


Af 
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QUESTION  105h  What  was  the  title,  position,  and  affiliation  of  the 
chairperson  of  the  formal  reviews? 

ANSWERS  Grouped  by  relationship  to  project 

a.  Project  Manager 
Manager  ADP 
Project  manager 
Computer  subsystem  manager 
Data  processing  manager 
Software  manager 
Group  leader  or  project  leader 
Project  manager  or  person  appointed  by  him 
SE/TD  project  manager 

b.  Senior  ADP  Manager 
Director 

Director  computer  programming 
Software  program  design  manager 
Department  head 
Director  to  chief 
Department  manager  for  software 
Software  divisica  manager;  reported  to  the  program  manager. 

c .  Senior  Non-ADP  Manager 
Director 

System  engineer  for  the  system  of  which  a  given  software 
product  was  part. 

d.  Customer 

Project  engineer  (customer) 

Customer  project  manager 

e .  Technical  Director 
Technical  director 
Technical  leader 
Systems  engineer 

Systems  engineer,  contractor  Assistant  Project  Manager 
Lead  software  engineer 
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f .  Corporate  Staff 

Vice  President  and  general  manager 

Management  analyst.  Integration/ control  office  of  mission 
group. 

g.  Varied 

Varied  -  reviews  generally  chaired  by  prime  contractor 
Varied 

Depends  upon  the  level  of  the  review 

Varied  from  vendor  representative  to  Director  of  Related 
Systems  functions 

h.  Review  Board 

Design  review  board  chairman,  systems  engineer 
Review  chairman 

Director,  Quality  Assurance  -  Reporting  to  company  president 
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QUESTION  109  Give  title,  position  and  affiliation  of  chairperson  of 
Configuration  Board. 

ANSWERS  Grouped  by  relationship  to  project 

a.  Project  Manager 
Program  manager 
Project  manager 
Project  leader 

Project  manager  assigned  a  senior  person^ 

Project  manager  of  project 

b .  Software  Project  Manager 
Computer  subsystem  manager 
Software  controller 

System  engineer  for  the  system  to  which  a  given  software 
product  was  a  part 

c.  System/Functional  Project  Manager 
Manager,  Systems  Engineering  (contractor) 

Deputy  for  Operations,  program  office 
Manager,  avionics  o'fficer 

System  project  engineer 
Director  of  Engineering 

Systems  engineer  for  the  system  of  which  a  given  software 
product  was  a  part 

Systems  Engineering  manager 

d.  Senior  Technician 
Chief  engineer 

e.  Corporation  Staff 
Mr.  John  Doe 

f .  Configuration  Management /Product  Assurance 
Software  product  assurance  manager 
Configuration  management  manager 
Configuration  control  board  representative 
Program  charge  review  board  chairman 

g.  Customer 

Project  engineer,  customer 

Joint  customer-supplier  chairmanship 
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QUESTION  115  If  it  were  within  your  power  to  make  any  changes  in  the 
method  or  procedures  followed  in  controlling  this  project  what  would 
these  changes  be,  or,  if  you  had  resources  available  to  undertake  research 
in  the  area  of  planning  which  aspect  would  you  explore? 

ANSWERS 

A  standing  review  committee  would  be  established  for  each  project 
with  critical  points  designated  for  technical  reviews. 

Don't  apply  -  T-specif ications  or  CSSR  to  software  development  project. 

Allow  time  for  an  orderly  approach  to  design  and  design  reviews. 

Establish  definite  packages  of  work  and  affect  regular  reviews  with 
milestones . 

Schedule  walk-throughs.  Hold  additional  formal  reviews. 

Provide  more  detailed  breakdown  of  task  and  associated  milestones, 
effort  estimates,  etc. 

Complete  top-down  design  with  walk-throughs. 

Keep  records  of  time  by  activity.  Document  changes  more  formally. 

More  formal  design  reviews  for  different  levels  of  software/engineering 
activities . 

Formalize  coding  standards  -  loosen  up  on  50  lines  of  code  per  program. 

Development  of  automated  procedures  would  be  of  assistance. 

More  tools. 

More  effective  ways  to  estimate  impact  (cost,  schedule)  on  new  change 
requests . 

Formal  configuration  management,  improved  software  library  and  link 
system. 

Earlier  configuration  control,  base-lined,  software  requirements. 

More  strict  adherence  to  base-line  with  schedule  modification  (or  at 
least  review)  for  all  changes  beyond  base-line. 

Would  minimize  involvement  of  the  agencies  and  departments  who  have 
the  capability  to  restrain  yet  do  not  have  potential  to  advance  develop¬ 
ment  (work)  efforts. 

Use  walk-throughs. 

Start  CM  procedures  -  for  this  project  it  started  the  second  time 
around  (design  context)  after  four  project  managers  had  their  chance  Co 
get  it  up. 

Aids  permitting  automatic  collection  of  development  status  data,  with¬ 
out  requiring  frequent  input  from  individual  contributors. 

Configuration  management  with  change  control,  baselines,  etc. 


Institute  formalized  project  control  procedures  including  full  range 
of  documentation  and  management  configuration  control  from  concept 
through  implementation  and  follow  on  maintenance. 


I 


I 


I 
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QUESTION  118  The  title  and  position  of  the  project  manager's  supervisor 
was : 

ANSWERS  Grouped  according  to  relationship  to  project  manager 

a.  A  Senior  ADP  Manager 

Chief,  Data  Automation  Branch 
Chief,  Digital  Applications 

Manager,  Software  Systems  Development  Department 
Operations  manager,  project  review  authority 
Operations  manager,  software  systems  operation 
Cijief,  Scientific  Applications  Branch 
Branch  head.  Programming  Techniques  Branch 
Software  Engineering  Division  leader 

Four  group  engineers  or  supervisors  in  charge  of  software 
engineering,  applications  software  systems  and  display, 
software  and  data  systems  test,  and  integration 

Director  -  site  facility 

Department  manager 

Department  head  -  systems  programming 

Director  computer  center 

Director,  software  development 

Department  manager  -  third  level  supervisor 

Assistant  director  -  Business  systems  development 

Director,  Operations 

Computer  system  officer 

b.  Senior  Non-ADP  Manager 
Operations  director 
Director  of  development 
Director  of  engineering 

Assistant  manager.  Engineering  Operations 
Manager,  Systems  Analysis  Department 
Avionics  chief  engineer 
Manager  engineering 

Considering  the  software  group  leader  as  project  manager,  his 
supervisor  was  Systems  Group  supervisor 

Senior  director 

Chief  engineer 
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b .  Senior  Mon-ADP  Manager 
Manager,  Avionics  Systems 
Section  manager 
Program/Component  manager 
Department  head 
Avionics  manager 

Branch  head 

System  project  engineer 

Project  leader 

Engineering  project  leader 

Project  engineer  -  development  manager 

Director 

Engineering  supervisor 
Manager  development  engineering 
Section  chief 
Section  manager 

c.  Program  Manager  (as  opposed  to  project  manager) 
Program  manager 

SPO  chief 

Matrix  -  Program  manager 
Computer  Laboratory  Manager 

d .  Corporation  Officer 
Program  Vice  President 
Vice  President 

Vice  President  Federal  Systems  Divisions 
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QUESTION  120  This  question  reports  the  title  and  position  of  people 
reporting  directly  to  the  project  manager.  However,  very  few  surveyees 
completed  the  POSITION  answer  of  the  Question.  It  was  felt  by  the  authors 
that  this  indicated  that  the  title  a  person  had  was  also  his  position. 


ANSWERS  This  is  the  only  place  the  clear  text  answers  to  Question  120A 
and  120B  appear.  The  tabulation  sheet  "blurs"  the  different  titles  used 
by  the  project  personnel.  The  answers  to  Question  120g  appears  in 
parenthesis  under  the  last  number. 


Posit:>on 
Section  mgr 
Section  mgr 
Section  mgr 

Section  supervisor 

Technical  Director 
Manager 

Analyst 

Programmer 

Typist 

Librarian 

Senior  Prog/Analyst 
Support  Supervisor 
Secretary 
Programmers 

Systems  engineers 
Hardware  engineers 
Software  engineers 

Senior  project  analyst 
Programmer  analyst 


Title 

Mgr,  Systems  Engr 
Mgr,  Programming 
Mgr,  Integration  i  Test 

Technical  Director 


Number 

1 

2 

1 

(4) 

1 

1 

(2) 

1 

4 

(5) 

3 

7 

1 

1 

(12) 

5 
1 
1 

12 

(20) 

5 

22 

3 

(30) 

1 

3 

(4) 
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Title 

Position 

Number 

Project  analyst  (ours) 

Chief  programmer 

3 

Project  analyst  (ours) 

Technical  coordinator 

1 

Progrannner/analyst  (ours) 

Team  programmer/analyst 

1 

Analyst  (cust) 

Team  programmer /analyst 

4 

Programmer  (cust) 

Team  programmer 

7 

(20) 

Senior  project  analyst 

2 

(17) 

Associate  engineer 

4 

Engineer 

6 

Senior  engineer 

2 

(12) 

Senior  engineer 

4 

Engineer 

7 

Associate  engineer 

3 

Librarian 

• 

1 

(15) 

Computer  hardware 

1 

Avionics  Interface 
hardware  design 

1 

Computer  SW  designer 

1 

(3) 

Software  engineering 

Manager 

20 

Software  development 

Manager 

35 

Computer  operators 

Manager 

30 

Plans/ controls 

Manager 

10 

/ 

(95) 

Engineering  specialist 

Level  25 

5 

Engineering  specialist 

Level  27 

2 

(8) 
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Title  Position  Number 

SW  development  1 

SW  Programmer/analyst  1 

SW  Configuration  mgn  1 

SW  engineering  1 

Administrator  1 

(5) 

Senior  analyst  1 

Programmer  2 

Engineer  2 

(5) 

Task  leader  2 

Engineer  2 

Programmer  2 

(6) 

Assistant  Project  manager  8 

Secretary  4 

Ci2) 

Mathematician  Analyst/ programmer  1 

Mathematician  Administrator  1 

Mathematician  Tester  1 

(4) 

Head  SW  engr  (ours)  1 

Head  SW  engr  (Contractor)  2 

Programmers  3 

Librarian  1 

Analyst  1 

(8) 

Managers  6 

Depucv  director  2 

Staff  assistants  4 

(12) 

Technical  director  1 

Team  chief  7 

(8) 


A 
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Title 

Position 

Number 

Program  analyst 

10 

(10) 

Engineers 

25 

(25) 

Group  engineer 

SW  engineering 

1 

Group  engineer 

Applications  SW 

1 

Group  engineer 

Sys  analysis  &  display  SW 

1 

Group  engineer 

Data  sys  test  &  integration 

1 

Secretary 

Secretary 

1 

(6) 

Software  requirements 

Supervisor 

6 

Flight/Mission  SW  design 

Supervisor 

11 

Automatic  test  equip 

SW  design 

Supervisor 

10 

Software  product  assurance 

Supervisor 

8 

(35) 

Manager  -  system 

1 

Manager  -  application 

1 

Manager  -  verification 

1 

Manager  -  program  control 

1 

(350) 

Data  anil/sts 

10 

(10) 

Programmer  analysts 

10 

(10) 

Junior  programmer 

1 

Programmer 

1 

a. 

Asst  Engineer /Programmer 

1 

Assoc  Engineer/Programmer 

1 

Engineer/Programraer 

2 

(6) 

Functional  team  leader 

3 

Programmers 

7-9 

(10-12) 
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Title 

Position 

Number 

Analyst 

6 

Programmer 

2 

Lab  integrator 

4 

Aircraft  operations 

3 

(20) 

Design  head 

1 

Programming  head 

1 

Test  team  head 

1 

Chief  analyst 

1 

(4) 

Project  SW  manager 

Line  supervisor 

15 

Test  engineer 

Line  supervisor 

15 

System  engineering 

Line  supervisor 

10 

(50) 

Senior  engineer 

4 

Specialist 

1 

(15) 

Member  Tech  Scaff  3 


Section  heads 
Group  heads 
Technical  staff 
Programmer /librarians 

Secretary 

Programmers 


Test/ integrators 
Project  manager 


Member  tech  staff 

Member  tech  staff 
Member  tech  staff 


(4) 

3 

8 

3 

41 

(55) 

0 

3  teams 
of  2,  3, 

4  people 
each 

3 

1 

(15) 
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Title  Position  Number 

Programmer /analysts  20 

(20) 

Group  leader  3 

(12) 

Supervisor  Applications  supervisor  1 

Hardware  analyst  HW  evaluation  &  install  2 

Data  base  analyst  Install  data  base  design  2 

(35) 

Gp  leader  -  applications  1 

Gp  leader  -  systems  programmer  1 

Gp  leader  -  HW  engineer  1 

Gp  leader  -  Standard  systems  1 


(approx  32) 


Programmer 

Analysts 

Computer  specialists  (100+) 

Deputy  Project  manager  1 

Task  managers  6 

Project  secretary  1 

(8) 

Deputy  project  manager 
SW  Division  manager 


Systems  Engineer  manager 
Operations  manager 

(50) 

4 

5 

(16) 


Computer  specialist 
Computer  specialist 


GS-12 

GS-11 
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Title  Position.  Number 

Senior  programmer  1 

Advisor  programmer  1 

Staff  programmer  6 

Senior  associate  programmer  4 

Advisor  analyst  2 

Staff  analyst  9 

(29) 
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QUESTION  127  If  you  had  it  within  your  power  to  implement  changes  in 
the  way  this  project  was  directed,  or  had  the  resources  to  devote  to 
research  in  this  area,  what  action  would  you  take? 

ANSWERS 

Greater  delegation  of  authority,  responsibility.  Clearer  assignment 
of  responsibility. 

Obtain  a  better  match  between  the  experience  of  the  personnel  and  the 
requirements  of  the  project. 

Perform  basic  research  in  software  lifecycle  cost. 

Less  centralized  control  by  the  project  manager. 

Increase  control/interaction  with  functional  group  developing  the 
company  proprietary  software. 

Delegate  more  to  technical  subordinates. 

More  resources  devoted  solely  to  highest  level  project  management. 

Would  add  more  formality  to  the  line  organization. 

Assume  a  knowledgeable  and  competent  project  manager  was  assigned 
during  the  entire  project  life. 
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QUESTION  133  What,  if  anything,  could  the  project  manager  have  done  to 
improve  his  ability  to  meet  the  schedule? 

ANSWERS 

Insisted  on  a  formalized  system  of  changes. 

Firmly  establish  methodology/design  before  implementation. 

Have  authority  over  all  phases  of  the  program,  not  just  software. 

Insisted  on  firm  baseline. 

Insist  on  firm  baseline  after  the  requirements  phase. 

Nothing  -  requirements  changes  are  vehicle  orientated  not  software. 

Have  a  better  estimate  of  the  job  initially. 

Use  basic  management  techniques  and  employ  more  $  and  people  -  early. 

Better  original  estimate.  Negotiate  schedule  change  with  each  require¬ 
ment  change. 

Restrict  number  of  modifications  and  suggestions  allowed. 

Enforce  structured  code  walk-through  earlier. 

Bid  more  manpower,  in  particular  in  software  documentation  to  meet 
milestone  -  Progressive  build  up  of  the  software  product  specification. 

-An  earlier  agreement  on  the  definition  of  system  requirements. 

Improve  support  received  from  ocher  organizations  -  on  time  delivery 
of  target  computer  and  hardware  test  beds. 

Should  have  had  formal  documentation  milestones/software  management 
plan. 

Stick  with  basic  requirements  except  where  a  change  is  required  to 
meet  basic  requirements.  Changes  to  basic  requirements  should  be 
reviewed  for  schedule  impact  instead  of  being  done  with  much  O.T. 

Minimize  number  of  changes  and  design  objectives. 

Overstaff  initially  to  provide  trained  cadre  for  the  accelerated 
schedule  to  come  later. 

Delay  other  projects. 

Baseline  requirements,  improve  computer  access,  enforce  better 
design  quality  standards,  enforce  more  structured  programming  standards. 

Better  monitoring  and  application  of  resources. 

Better  monitoring  and  testing. 

Respond  to  the  early  warning  signals  with  more  aggressiveness  and 
alacrity. 

Improve  test  approach,  better  control  of  project  resources,  increased 
involvement  of  true  user  community,  better  control  over  all  elements  of 
the  project  (i.e.,  company  developed  proprietary  package). 


AD-A117  998 
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Little  requirements  or  de-scope  implementation. 

Good  software  engineering  techniques  were  not  used  the  first  time 
around . 

Baseline  requirements  and  ensure  that  the  user  is  involved  and  docu¬ 
ments  the  proper  test  procedures. 


<r- 

4^ 
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QUESTION  137  What,  if  anything,  could  the  program  manager  have  done  to 
improve  his  ability  to  meet  the  budget? 

ANSWERS 

Provide  more  visibility  to  Che  impact  of  changes  in  order  to  reduce 
requests. 

Better  initial  estimate.  Firm  establishment  of  methodology  design 
before  implementation. 

Maintained  lower  staffing  profile  until  requirements  were  more  firm. 

Realism  on  the  part  of  the  customer  was  an  absolute  necessity  but  was 
not  a  reality. 

Better  estimates,  better  style,  better  subcontract. 

Nothing. 

Better  initial  estimate. 

Restrict  number  of  modifications  and  suggestions  allowed. 

Freeze  baseline. 

Bid  more  realistically. 

Bid  correct  cost  in  first  place.  (The  program  manager  at  the  top  can 
do  as  ours  did.  If  he  does  not  cut  too  deep,  one  element  may  overrun, 
but  whole  project  would  be  on  or  under  cost.)  Apply  modern  software 
design  techniques. 

Phase  staffing.  Plan  the  staffing  to  be  commensurate  with  the 
development  activities. 

Delete  documentation  requirements. 

Take  a  harder  approach  with  the  customer  to  make  his  requirements 
more  firm  and  more  unified. 

Allow  contingency,  indirect  rate  changes. 

Baseline  requirements  and  ensure  that  the  user  is  involved  in  and 
documents  the  proper  test  procedures. 

Limit  requirements  or  de-scope  implementation. 

Update  projections  based  on  experience  rather  than  wishes. 

Closer  review  of  design. 

Better  accounting  procedures,  baseline  requirements,  improve 
computer  access,  enforce  better  design  quality  standards,  enforce  more 
structured  programming  standarus. 

Earlier  definition  and  freezing  of  requirements.  More  pressure  on 
customers  to  supply  requirements  and  limit  them. 

Act  more  aggressive  toward  demanding  test  plan  earlier  and  to  a 
detailed  level. 


more 
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Improve  test  approach,  better  control  of  project  resources,  increased 
involvement  of  true  user  community.  Better  control  over  all  elements  of 
the  project  (i.e.,  company  developed  proprietary  package). 

Delayed  staffing,  thereby  missing  initial  milestones  which  later 
proved  to  be  misdirected. 
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QUESTION  148  Was  the  delivered  system  useable?  If  so,  how  was  this 
determination  made? 

ANSWERS 

Yes  -  After  fleet  training,  system  went  into  operation  and  replaced 
previous  system. 

It  was  deemed  useable  by  a  complete  user  evaluation  and  test  at  his 
site. 

Yes  -  Company  and  customer  demonstration  during  development  phase. 

Accepted  by  user  as  operational. 

In  operation,  supporting  system  evaluation  and  fleet  exercises  since 
1974. 

Yes  -  test. 

Presently  being  determined. 

System  has  been  used  in  demonstrations  but  not  delivered  to  users. 

Capable  of  performing  task  for  which  it  was  designed. 

Technical  and  operational  testing  in  the  field. 

System  not  yet  delivered. 

Not  yet  completed. 

[it]  Is  in  production;  [it]  is  being  used  by  non-computer  types  on 
daily  basis. 

Yes  -  operational  field  tests. 

Not  yet  delivered. 

Through  use. 

Real  system  tests  using  live  data.  Data  evaluation  showed  system 
met  requirements. 

Ability  to  check  out  and  fly  the  vehicle  without  conscious  effort  to 
"deal  with"  the  problem. 

Combined  DT&E/IOT&E. 

Flight  test. 

The  user  ran  the  system. 

Demonstrating  the  use. 

Passed  the  designed  tests. 

Independent  testing  by  several  agencies  before  delivery  followed  by 
monitoring  of  full  use. 

Yes,  through  testing  by  Independent  contractor  and  operational  usage. 

Yes  -  review  of  operational  utilization. 


387 


No,  did  not  respond  fast  enough  to  user  demands. 

Yes,  demonstrated  integrated  operation  in  complete  weapon  delivery 
system. 

It's  still  in  use. 

Availability  factor  to  support  active  flight  test  programs. 

Yes,  it  satisfies  the  user  (requirements). 

Yes,  it  was  subjected  to  an  operational  test  by  the  user. 

Presently  being  used. 

Yes,  continued  use  by  customer. 

Flight  test. 

Satisfying  the  avionics  integration  requirements  -  hardware  and  soft¬ 
ware  working  together. 

Through  testing  and  verification. 

Not  determined. 

Turn  key  system  used  for  factory/depot  testing  designed  for  manu¬ 
facturer,  machine  interface,  etc. 

Customer  field  tests. 

Demonstration. 

Through  operation. 

Field  exercises  in  competition  with  another  customer's  systems. 
Customer  was  satisfied. 

Yes,  used  for  both  launches  -  but  painfully! 

System  was  usable,  but  not  up  to  expectations  of  users. 

The  user  group  used  for  daily  operations.  Prior  systems  were 
scrapped  day  I. 

Still  in  final  system  acceptance  testing. 

By  the  users  "real"  application,  immediately  after  delivery. 

Product  test  with  user  involvement. 

The  jury  is  still  out. 


( 
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QUESTION  159  Give  some  lessons  learned  from  this  project. 

ANSWERS 

Delegate  responsibility  and  authority:  maintain  communications. 

The  interface  with  an  IV  and  V  contractor  is  much  more  time-consuming 
than  estimated.  Customer  interface  at  a  detailed  level  is  costly. 

Must  have  well  defined,  fall-back  position  to  go  to  if  problems 
develop.  Must  hold  line  on  new  suggestions  and  changes  introduced. 

Make  sure  development/test  tools  are  ready;  automate  the  process  as 
much  as  possible. 

Establish  requirement  clearly  before  designing;  establish  design 
clearly  before  programming.  Give  more  attention  to  planning  and  effort/ 
cost  estimates.  Need  greater  documentation  of  decisions,  specifications, 
and  design. 

Careful  building  of  requirement  specifications  to  facilitate  testing 
pays  off,  configuration  control  pays  off,  integrated  engineering/ 
programming  team  pays  off. 

Must  have  firm  base  line;  good  documentation  of  higher  levels  of 
effort  must  precede  lower  levels  of  effort. 

Sound  management  (a  la  hardware  projects)  works  well  with  software 
projects. 

Streamline  requirements  before  software  development. 

Good  foresight  and  planning  very  essential. 

Good  management  goes  a  long  way.  Do  front  end  work  well,  get  a  good 
sub-contractor  and  good  sub-contract.  Develop  good  management 
visibility  too.  . 

You  assume  you  know  the  project  managers  main  goal!!  in  Question  158. 
[To  deliver  on  time,  within  budget,  and  meeting  the  requirements  of  the 
system,  where  the  final  software  product  is  reliable,  maintainable,  and 
useable. ]  If  the  contract  has  been  properly  written  (an  incentive 
contract),  the  project  managers  objective  is  to  maximize  profit!  If  he 
does,  the  customer  should  be  happy  because  the  contract  will  have 
steered  the  project  manager  and  the  customer  wrote  the  contract!  If 
there  is  no  requirement  that  the  software  be  reliable,  maintainable 
and/or  useable,  the  project  manager  should  not  have  these  as  a  major 
goal.  On  time,  yes!  Within  budget,  yes!  Meeting  requirements,  yes! 

But  more  than  that  not  necessary.  Sometimes  though,  contracts  are  not 
written  to  reflect  the  true  desires  of  the  customer.  Therefore,  when 
project  manager  goes  to  maximize  profits,  the  customer  is  not 
necessarily  happy!!! 

1.  Documentation  and  flow  charting  more  expensive  than  thought.  2. 

The  average  software  designer  fails  to  see  the  need  for  documentation  or 
discipline  until  job  is  near  completion.  3.  Documentation,  discipline, 
good  configuration  management,  sound  planning,  detailed  schedule 
commitments,  software  design  standards,  error  tracking,  top-down  design 
and  combined  top— down/bo ttom— up  testing  pays  off. 
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More  strict  monitoring  and  enforcement  of  standards  and  roles  is 
needed . 

Enhanced  programmer  professionalism  is  extremely  important.  Need 
better  reporting  mechanism.  Need  automated  means  of  monitoring 
programming  and  design  progress. 

1.  Require  intense  interface  with  user  community.  2.  Increas-r  the 
planning  phase.  3.  Complete  detail  design  prior  to  code  initiation. 

4.  Require  full  system  walk-throughs.  5.  Increase  formalization  of 
planning.  6.  Do  not  build  test  team  from  original  project  personnel  to 
transition  to  a  testing  function.  7.  Dedicate  project  teams  to  individ¬ 
ual  projects.  8.  Implement  a  separate  test  function  at  project  initi¬ 
ation.  9.  Create  a  separate  documentation  function.  10.  Have  document¬ 
ation  performed  by  other  than  programmers.  11.  Keep  personnel  on 
project  for  duration  of  project.  Do  not  remove  them  even  to  put  out 
brush  fires.  12.  Improve  communication  between  lower  level  management 
and  team  member  level.  13.  Require  everyone  to  comply  with  project 
controls.  14.  Complete  test  plan.  15.  Have  better  control  of  project 
resources. 

1.  Requirement  maturity  less  than  originally  stated.  2.  Development 
tool  (including  facility)  inadequately  specified.  3.  Computer  memory 
insufficient  to  satisfy  the  computation  load.  4.  Staffing  plan  not 
consistent  with  development  activities. 

1.  Highly  motivated  inexperienced  staff  can  consistently  out  perform 
less  motivated  senior  staff.  2.  Never  believe  anything  a  computer 
vendor  says. 

1.  Keep  software  test  in  the  software  project  team.  2.  Use  rate 
charting  to  monitor,  include  status  -  on  manual  basis.  3.  Have  good 
standards  and  conventions.  4.  Develop  support  software  early.  5.  Have 
software  project  control  facility  and  facility  scheduling.  6.  Keep 
close  coordination  between  test  and  programming  groups. 

Use  standard  vendor  supply  operating  system.  Use  of  high  level 
language,  more  walk-throughs,  more  formal  specifications. 

Be  sure  the  system  design  is  feasible  and  practical  before  beginning 
software  design. 

1.  Independent  test  team  essential.  2.  Develop  sizing/ timing  budget 
for  modules  and  monitor  and  maintain. 

Need  aids  and  support  librarians. 

Proximity  of  laboratory,  computer  facility,  test  bed  and  people  to 
each  other  is  the  key. 

A  detailed  computer  program  development  plan  (CPDP)  must  be  completed 
early  (within  30  days  after  contract  award) . 

Documentation  must  be  developed  during  the  computer  program  develop¬ 
ment.  This  takes  manpower  and  time  which  was  not  planned  and  allocated. 

Require  that  software  is  tested  by  a  separate  group. 
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Peer  reviews  should  be  used. 

1.  Politics  plays  a  more  significant  role  than  pure  technical 
consideration.  2.  Even  the  presence  of  a  few  super  stars  on  the  team 
does  not  guarantee  success. 

Pick  team  carefully,  insist  on  full  range  of  documentation,  employ 
configuration  control,  use  walk-throughs,  estimate  high  -  expect  the 
worst,  develop  accurate/reliable  reporting  techniques.  Control  the 
project. 

Need  to  spell  out  specifications  system,  requirements  with  CM 
procedures,  change  control,  baselines;  more  user  involvement,  product 
test  plan  by  user. 

Detail  requirements  definition;  code  walk-throughs;  early  test 
planning;  extensive  engineering  documentation:  very,  very  important. 

Do  not  let  customer  dictate  schedule,  design  and  configuration  with¬ 
out  contractor  concurrence  and  agreement. 

Need  to  spell  out  specification  requirements  with  configuration 
management  procedures,  change  control,  baseline,  more  user  involvement, 
product  test  plan  by  user.  Scheduling  around  holiday  causes  delay. 
"Programmer  is  a  programmer  is  a  programmer"  isn't  true.  Extensive  time 
away  from  home  costly  and  hard  on  personnel  with  compensatory  time 
higher  than  needed. 
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QUESTION  164  (Formerly  Question  25  of  PART  THREE  of  original  survey 
form)  Please  furnish  any  additional  comments  or  statements  concerning 
this  survey  or  the  science  of  software  engineering  project  management. 

ANSWERS 

Hope  you  get  many  completions,  hope  you  can  digest  this,  good  list  of 
though  items.  Suggest  followup  interviews  with  selected  submitters  to 
get  "personality"  of  company/pro j ect . 

There  appears  to  be  an  over  emphasis  upon  quantifications  of 
subjective  functions.  It  is  more  important  to  define  visibility  methods 
and  automated  procedures  to  keep  programmer  ingenuity  out  of  straight¬ 
forward  program  design. 

Software  engineering  management  differs  very  little  from  the  R&D  or 
prototype  development  of  hardware.  Unfortunately  there  is  a  tendency  to 
compare  software  development,  which  has  no  real  production  (repetitive) 
phase  with  the  production  phase  of  hardware  development.  Software 
development  is  a  design  and  test  effort. 

Many  of  the  questions  asked,  especially  in  Part  3,  are  philosophic  in 
nature.  Software  engineering  is  still  at  issue  as  a  discipline.  I 
believe  that  many  approaches  to  managing  software  are  valid,  and  that 
many  problems  result  from  [failures  in] :  1)  the  application  of  uniform 

standards,  and,  more  important,  2)  the  commitment  to  enforcing  such 
standards.  If  this  results  in  additional  initial  costs  during  the 
development  of  projects  the  savings  will  off-set  that  cost,  many  fold 
over  the  life  of  the  system. 

Software  development  is  a  very  tough  management  problem.  It  isn't 
clear  whether  formal  techniques  are  a  significant  aid,  and,  if  so, 
whether  they  justify  their  overhead.  Honest,  published  case  studies 
addressing  those  questions  are  needed.  Time  and  effort  estimates, 
especially,  are  a  problem.  Software  projects  are  chronically  subject  to 
cost  overruns  and  schedule  overruns,  or  altarnatively ,  failure  to  fully 
meet  objectives.  How  can  this  situation  be  overcome?  I  found  it 
difficult  to  relate  much  of  the  survey  to  the  particular  project  I  was 
addressing.  Primarily,  this  was  because  the  survey  presupposed  orderly 
progress  from  a  clear  starting  point  to  delivery.  The  project  didn't 
(and  probably  couldn't  have)  gone  that  way. 

Seems  to  be  a  universal  difference  between  the  way  things  are  and  the 
way  they  ought  to  be.  Primary  case  in  point  is  the  extensive  modific¬ 
ations  necessary  in  most  specifications  subsequent  to  their  initial 
development.  This  has  a  ripple  effect  on  software  development  down 
stream.  Until  such  time  as  the  perfect  specifications  writer  is  born, 
some  small  part  of  the  "science  of  software  engineering"  will  remain  an 
art,  and  hence,  undefined  and  unmeasureable.  We  should  probably  accept 
that,  and  realize  that  all  decisions  cannot  be  made  on  the  basis  of  100 
percent  accurate  information.  The  survey  itself  was  exhaustive  and 
exhausting.  It  will  be  interesting  to  see  if  general  conclusions  can  be 
drawn  from  these  questions,  particularly  from  projects  which  do  not 
necessarily  "fit  the  mold".  Good  luck. 


This  data  should  be  collected  by  interview  instead  of  a  standard 
form  due  to  the  varying  organizations  and  projects. 

The  answers  to  this  questionnaire  are  based  on  a  by-gone  project,  and 
do  not  necessarily  represent  the  way  [we  are]  now  managing  and  develop¬ 
ing  software. 

I  wonder  how  many  souls  had  the  patience  to  go  through  this  question¬ 
naire.  Good  luck! 


The  following  are  extracts  from  correspondence  received  by  the  authors 
from  the  Surveyees. 

Correspondence  Extract  1 


We  had  a  bit  of  a  problem  in  trying  to  construct  answers  which  would 
be  meaningful  and  reflect  the  true  state  of  affairs  at  our  Division. 

We  are  a  Division  of  a  Parent  Company.  Consequently,  we  can  be 
viewed  as  a  corporate  entity,  as  a  division,  or  as  part  of  the  total 
corporate  entity.  I  chose  to  regard  ourselves  as  the  total  entity  for 
purposes  of  answering  this  questionnaire.  The  only  exception  to  this  is 
the  answer  to  one  question  in  which  we  have  a  corporate  systems  group 
which  are  used  in  evaluating  any  purchased  software.  This  is  a  response 
to  Question  32,  Part  I. 

Another  difficulty  which  I  encountered  was  interpretation  of  whether 
the  questions  should  be  addressed  only  to  what  we  call  our  data  processing 
group  who  are  responsible  for  the  general  divisional  computer  use.  It 
is  this  group  which  generally  writes  the  business  programs,  e.g.,  pay¬ 
roll,  inventory,  bill  of  materials.  In  addition  our  product  involves 
the  computer  in  essentially  every  case.  Consequently,  we  have  a  large 
body  of  people  who  program  computers  for  the  product  which  we  sell.  In 
order  to  accommodate  these  two  diverse  activities  I  have  had  one 
questionnaire  filled  out  by  data  processing  to  reflect  activities  for 
which  they  are  responsible  and  the  other  filled  out  by  different  people 
dealing  with  the  actual  product.  Typically  we  may  refer  to  what  they 
are  doing  as  applications  programming. 

A  further  complication  arises  in  Chat  we  have  a  group  of  software 
specialists  who  write  special  software  in  support  of  the  applications 
programmers.  These  people  write  edits,  debug  routines,  maintenance  and 
test  routines,  library  routines  and  the  like.  Their  work  gets  very 
similar  to  Che  kinds  of  programs  that  we  would  generally  be  buying  for 
the  data  processing  group  from  outside  sources. 

As  you  can  see  we  are  somewhat  complex  and  interpreting  our  data  may 
be  difficult  for  that  reason. 

Correspondence  Extract  2 

Just  a  note  -  thank  you  for  the  opportunity  Co  respond.  Some  comments 
follow: 

The  questionnaire  was  very  difficult  to  answer  in  some  areas  -  inter¬ 
preting/applying  the  question  to  our  project  was  often  frustrating. 

The  emphasis  was  on  ADP  -  but  I  believe,  for  real-time  systems  -  that 
the  true  significance  has  been  missed.  The  thought  is  this:  the 
engineering  has  now  moved  into  the  computer,  rather  than  being  realized 
as  discrete  logic,  etc.  Your  "analysts"  may  be  skilled,  but  it  can't 
be  done  without  Che  software-knowledgeable,  system-knowledgeable 
engineer . 
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My  encouragement  to  you  on  this  survey  and  the  consolidation  of 
responses. 

Correspondence  Extract  3 

As  I  told  you  in  my  phone  call  I  am  seriously  concerned  that  the 
answers  to  this  questionnaire  may  be  misleading.  The  questionnaire  seems 
to  me  to  embody  a  number  of  assumptions  about  organization,  personnel, 
staffing,  work  flow,  responsibility,  which  are  applicable  to  a  tradi¬ 
tional  data  processing  center,  but  which  are  less  appropriate  -  or  even 
wrong  -  in  the  effective  engineering  uses  of  embedded  computers.  At 
least  we  do  not  do  things  these  ways  (.  .  .  .].  I  am  concerned  because 
the  thrust  of  new  DoD  directives  and  policies  is  in  the  direction  of 
concepts  that  frankly  I  consider  obsolescent  and  inappropriate. 

Correspondence  Extract  4 

Enclosed  please  find  the  completed  survey  forms.  We  apologize  for 
our  delay  in  responding,  but  the  forms  arrived  while  the  primary 
respondents  were  on  vacation. 

.  .  .  two  programs  [were]  reported  on. 

[The  first]  project  was  characterized  by  the  judicious  introduction 
of  advanced  software  techniques  and  very  strong  project  management  and 
reviewing  practices.  (As  an  example  of  the  latter,  the  software  was 
treated  as  hardware  under  the  same  configuration  control  system.) 

In  summary,  the  computer  systems  were  delivered  on  schedule  and 
slightly  under  budget.  After  two  years  of  continuous  use  very  few  soft¬ 
ware  errors  have  been  detected. 

The  [second]  project  was  one  where  almost  no  advanced  software  or 
management  techniques  were  used,  and  yet  the  program  culminated  with  a 
successful  .  .  .  test  .  .  .  within  budget  and  on  schedule. 

In  conclusion,  we  hope  these  responses  reach  you  in  tine  to  be  of  use 
and  we  look  forward  to  meeting  with  you  and  seeing  the  results  of  the 
overall  survey  at  the  AIAA  Conference. 

Correspondence  Extract  5 

In  response  to  the  referenced  letter,  we  have  enclosed  three 
different  completed  copies  of  your  software  development  questionnaire 
marked  (A) ,  (B)  and  (C) .  Each  of  the  questionnaires  was  completed  by 
different  group  supervisors  in  our  Programming  Development  Section. 

.  .  .  Their  responses  are  candid,  their  own  and  no  higher  level 
organizational  image  polishing  has  been  applied. 

Your  questionnaire  aroused  much  interest  .  .  .  and  numerous  copies 
of  the  questionnaire  were  distributed  for  evaluation,  comment  and  dis¬ 
cussion.  Needless  to  say  this  Section  has  an  excellent  record,  but  they 
are  continually  seeking  improved  techniques  for  increasing  the  overall 
productivity  of  their  software  development  teams.  The  persons 
completing  the  questionnaire  believed  that  the  exercise  was  fairly  time- 
consuming,  sort  of  soul-searching,  but  very  worthwhile. 


I 
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Correspondence  Extract  6 

I  greatly  regret  the  delay  in  providing  the  enclosed  data  which  was 
promised  to  you  for  a  much  earlier  date.  In  part,  interestingly 
enough,  this  delay  relates  to  the  subject  of  the  survey  in  that  we  have 
been  examining  our  organization's  approach  to  software  development  and 
are  in  the  process  of  revamping  both  the  organizational  structure  and 
the  methods  and  procedures  used  in  connection  with  the  development  of 
embedded  software.  We  expect  by  this  means  to  improve  both  the  quality 
of  our  product  and  our  efficiency  in  producing  the  high  quality  software 
our  customer's  require.  Again,  please  accept  my  sincere  apology  for  the 
extended  delay.  Thank  you. 

Correspondence  Extract  7 

I  have  struggled  through  the  questionnaire  and  filled  it  out  as  best 
as  possible.  It  seems  to  be  aimed  at  the  big  project  with  lots  of 
people,  lots  of  management,  documentation,  configuration  control,  etc. 
These  are  all  things  which,  in  my  judgement,  guarantee  failure  and  over¬ 
runs  (usually  both).  I  have  become  convinced  that  the  only  way  to  manage 
complex  software  projects  is  with  a  small  group  (not  more  chan  ten)  of 
competent  people.  If  Che  project  is  large,  it  must  be  developed  incre¬ 
mentally. 

Telephone  Call  Extract  8 

Mr.  "Smith"  called  concerning  the  survey.  Eie  thought  that  the 
survey  was  super,  well-liked  by  all  of  them  at  his  organization,  and  he 
asked  permission  to  use  it  as  internal  survey  to  determine  how  their 
developments  were  going.  Apparently  Mr.  "Smith"  and  his  Corporation 
need  some  means  of  keeping  software  project  history.  They  view  our 
survey  as  a  means  of  doing  this. 

Correspondence  Extract  9 

Here  it  is  at  long  last  -  warts  and  all.  Good  luck. 


